Can someone please help me figure how to build on linux ?

[yassinm]

This page is what i am following.

• It would be nice if a section was avalible for people building in linux as well :-)
• Also it does not say what would you expect in a successful run

Here is my current environment in my ubuntu machine

$go version
go version go1.17.5 linux/amd64

$cmake --version
cmake version 3.22.1

CMake suite maintained and supported by Kitware (kitware.com/cmake).

$echo $GOPATH 
/home/ymo/3rdp/dev/golang/releases/golatest-packages

$bazelisk version
Bazelisk version: development
Build label: 4.2.2
Build target: bazel-out/k8-opt/bin/src/main/java/com/google/devtools/build/lib/bazel/BazelServer_deploy.jar
Build time: Thu Dec 2 18:15:58 2021 (1638468958)
Build timestamp: 1638468958
Build timestamp as int: 1638468958

Here is what i am getting when i do make build

$cd ~/3rdp/dev/golang/releases/golatest-packages/src/github.com/cockroachdb/cockroach

$make build
Running make with -j8
GOPATH set to /home/ymo/3rdp/dev/golang/releases/golatest-packages
Warning: 'ccache' not found, consider installing it for faster builds
Warning: 'bazel' not found (`brew install bazel` for macs)
installing githooks/commit-msg
installing githooks/post-checkout
installing githooks/post-merge
installing githooks/post-rewrite
make: *** No rule to make target 'vendor/modules.txt', needed by 'bin/.bootstrap'.  Stop.
make: *** Waiting for unfinished jobs....
installing githooks/prepare-commit-msg

Notice how it does not say what is failing. Any help would be much appreciated !

[msirek]

Hi @yassinm, it looks like Bazelisk is missing on your system (2nd from the bottom in the prerequisite list):
Warning: 'bazel' not found (brew install bazel for macs)

I think if you install Bazelisk you will be one step closer to getting the build to work. Good luck!!

[yassinm]

@msirek oh sorry I already have bazelisk installed i forgot to put that in the env ! sorry!!!

[yassinm]

I installed bazelisk using the go install method mentioned here this does not put bazel in the path by default. I tried symlinking bazelisk to bazel and this is what i am getting now.

$make build
Running make with -j8
GOPATH set to /home/ymo/3rdp/dev/golang/releases/golatest-packages
Warning: 'ccache' not found, consider installing it for faster builds
installing githooks/commit-msg
installing githooks/post-checkout
installing githooks/post-merge
installing githooks/post-rewrite
make: *** No rule to make target 'vendor/modules.txt', needed by 'bin/.bootstrap'.  Stop.
make: *** Waiting for unfinished jobs....
installing githooks/prepare-commit-msg

[dt]

While you need Bazel / bazelisk for development work (e.g. to regenerate BUILD files if you make code changes and want to submit them in a PR), you shouldn't need it just to get that make build mentioned here to work. It looks like we're hitting a different error. related to a file in the vendor, which makes me wonder if the submodules -- which includes vendor/ -- didn't initialize correctly?

[dt]

Can you try running git submodule update --init to ensure the submodules are initialized and see if that helps?

[yassinm]

@dt I did that and it somewhat made the build pass the previous error. At this point i have to keep running make build it keeps failing in no consistent way. As such, I am still not able to do a full build with one command. But at least i am able to run some tests in vscode and looking at the go code. So at least i am not blocked.

Thank you so much for your help folks !!!

[yassinm]

HEre is one such error

[BABEL] Note: The code generator has deoptimised the styling of /home/ymo/3rdp/dev/golang/releases/golatest-packages/src/github.com/cockroachdb/cockroach/pkg/ui/ccl/src/js/protos.js as it exceeds the max of 500KB.


Hash: 45ff7df36117f9614191
Version: webpack 4.41.5
Time: 39871ms
Built at: 01/14/2022 1:27:42 AM
            Asset      Size  Chunks             Chunk Names
protos.ccl.dll.js  1.95 MiB       0  [emitted]  protos
Entrypoint protos = protos.ccl.dll.js
[4] dll protos 12 bytes {0} [built]
[5] ./ccl/src/js/protos.js 5.83 MiB {0} [built]
[7] (webpack)/buildin/global.js 475 bytes {0} [built]
    + 18 hidden modules
Hash: abe4d3349f243e1db02e
Version: webpack 4.41.5
Time: 63357ms
Built at: 01/14/2022 1:28:05 AM
            Asset      Size  Chunks                    Chunk Names
vendor.oss.dll.js  8.87 MiB       0  [emitted]  [big]  vendor
Entrypoint vendor [big] = vendor.oss.dll.js
  [48] (webpack)/buildin/global.js 472 bytes {0} [built]
 [246] (webpack)/buildin/module.js 497 bytes {0} [built]
[1029] dll vendor 12 bytes {0} [built]
[1038] (webpack)/buildin/harmony-module.js 573 bytes {0} [built]
[1134] ./node_modules/moment/locale sync ^\.\/.*$ 3 KiB {0} [built]
    + 1853 hidden modules

WARNING in asset size limit: The following asset(s) exceed the recommended size limit (3.81 MiB).
This can impact web performance.
Assets: 
  vendor.oss.dll.js (8.87 MiB)

WARNING in entrypoint size limit: The following entrypoint(s) combined asset size exceeds the recommended limit (3.81 MiB). This can impact web performance.
Entrypoints:
  vendor (8.87 MiB)
      vendor.oss.dll.js


WARNING in webpack performance recommendations: 
You can limit the size of your bundles by using import() or require.ensure to lazy load some parts of your application.
For more info visit https://webpack.js.org/guides/code-splitting/

[yassinm]

The build just stops for no particular reason. Here is where it stopped next time i ran make build again

ithub.com/cockroachdb/cockroach/pkg/sql/catalog/tabledesc
github.com/cockroachdb/cockroach/pkg/sql/catalog/systemschema
github.com/cockroachdb/cockroach/pkg/sql/catalog/catalogkeys
github.com/cockroachdb/cockroach/pkg/sql/catalog/catalogkv
github.com/cockroachdb/cockroach/pkg/sql/sem/builtins
github.com/cockroachdb/cockroach/pkg/cmd/docgen
go-bindata -pkg distccl -o pkg/ui/distccl/bindata.go -prefix pkg/ui/distccl pkg/ui/distccl/...
echo 'func init() { ui.Asset = Asset; ui.AssetDir = AssetDir; ui.AssetInfo = AssetInfo }' >> pkg/ui/distccl/bindata.go
gofmt -s -w pkg/ui/distccl/bindata.go
goimports -w pkg/ui/distccl/bindata.go
ymo@ymo-acer:~/3rdp/dev/golang/releases/golatest-packages/src/github.com/cockroachdb/cockroach$ 

[msirek]

It's possible the failure message is higher up in the output. Since make is invoked with -j8 as a parallel build on your system, what you see at the end of the output may be coming from some of the build tasks that did not hit the failure.
Perhaps running make -j1 build to use a single build task would show the failure closer to the end of the output. Typically on a successful build you would see the following line close to the end of the output:
github.com/cockroachdb/cockroach/pkg/cmd/cockroach

Make is being replaced with bazel (invoked with the dev command) for doing builds of cockroach, so you could do a make clean and then try building via ./dev build. This might give you a more useful error message for you to debug:

https://cockroachlabs.atlassian.net/wiki/spaces/CRDB/pages/2221703221/Developing+with+Bazel

[yassinm]

That works! Now i was able to find where the build was failing.

$make -j1 build
GOPATH set to /home/ymo/3rdp/dev/golang/releases/golatest-packages
rm -rf /home/ymo/3rdp/dev/golang/releases/golatest-packages/native/x86_64-linux-gnu/libedit
mkdir -p /home/ymo/3rdp/dev/golang/releases/golatest-packages/native/x86_64-linux-gnu/libedit
cd /home/ymo/3rdp/dev/golang/releases/golatest-packages/native/x86_64-linux-gnu/libedit && /home/ymo/3rdp/dev/golang/releases/golatest-packages/src/github.com/cockroachdb/cockroach/c-deps/libedit/configure   --disable-examples --disable-shared
checking for gcc... gcc
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables... 
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether gcc accepts -g... yes
checking for gcc option to accept ISO C89... none needed
checking whether gcc understands -c and -o together... yes
checking how to run the C preprocessor... gcc -E
checking for grep that handles long lines and -e... /usr/bin/grep
checking for egrep... /usr/bin/grep -E
checking for ANSI C header files... yes
checking for sys/types.h... yes
checking for sys/stat.h... yes
checking for stdlib.h... yes
checking for string.h... yes
checking for memory.h... yes
checking for strings.h... yes
checking for inttypes.h... yes
checking for stdint.h... yes
checking for unistd.h... yes
checking minix/config.h usability... no
checking minix/config.h presence... no
checking for minix/config.h... no
checking whether it is safe to define __EXTENSIONS__... yes
checking for a BSD-compatible install... /usr/bin/install -c
checking whether build environment is sane... yes
checking for a thread-safe mkdir -p... /usr/bin/mkdir -p
checking for gawk... no
checking for mawk... mawk
checking whether make sets $(MAKE)... yes
checking whether make supports the include directive... yes (GNU style)
checking whether make supports nested variables... yes
checking dependency style of gcc... gcc3
checking build system type... x86_64-pc-linux-gnu
checking host system type... x86_64-pc-linux-gnu
checking how to print strings... printf
checking for a sed that does not truncate output... /usr/bin/sed
checking for fgrep... /usr/bin/grep -F
checking for ld used by gcc... /usr/bin/ld
checking if the linker (/usr/bin/ld) is GNU ld... yes
checking for BSD- or MS-compatible name lister (nm)... /usr/bin/nm -B
checking the name lister (/usr/bin/nm -B) interface... BSD nm
checking whether ln -s works... yes
checking the maximum length of command line arguments... 1572864
checking how to convert x86_64-pc-linux-gnu file names to x86_64-pc-linux-gnu format... func_convert_file_noop
checking how to convert x86_64-pc-linux-gnu file names to toolchain format... func_convert_file_noop
checking for /usr/bin/ld option to reload object files... -r
checking for objdump... objdump
checking how to recognize dependent libraries... pass_all
checking for dlltool... no
checking how to associate runtime and link libraries... printf %s\n
checking for ar... ar
checking for archiver @FILE support... @
checking for strip... strip
checking for ranlib... ranlib
checking command to parse /usr/bin/nm -B output from gcc object... ok
checking for sysroot... no
checking for a working dd... /usr/bin/dd
checking how to truncate binary pipes... /usr/bin/dd bs=4096 count=1
checking for mt... mt
checking if mt is a manifest tool... no
checking for dlfcn.h... yes
checking for objdir... .libs
checking if gcc supports -fno-rtti -fno-exceptions... no
checking for gcc option to produce PIC... -fPIC -DPIC
checking if gcc PIC flag -fPIC -DPIC works... yes
checking if gcc static flag -static works... yes
checking if gcc supports -c -o file.o... yes
checking if gcc supports -c -o file.o... (cached) yes
checking whether the gcc linker (/usr/bin/ld -m elf_x86_64) supports shared libraries... yes
checking dynamic linker characteristics... GNU/Linux ld.so
checking how to hardcode library paths into programs... immediate
checking whether stripping libraries is possible... yes
checking if libtool supports shared libraries... yes
checking whether to build shared libraries... no
checking whether to build static libraries... yes
checking whether make supports nested variables... (cached) yes
checking whether to enable maintainer-specific portions of Makefiles... no
checking for gcc option to accept ISO C99... none needed
checking whether ln -s works... yes
checking for gawk... (cached) mawk
checking for nroff... /usr/bin/nroff
checking for tgetent in -lncurses... no
checking for tgetent in -lcurses... no
checking for tgetent in -ltermcap... no
configure: error: libncurses, libcurses, or libtermcap is required!
make: *** [Makefile:694: /home/ymo/3rdp/dev/golang/releases/golatest-packages/native/x86_64-linux-gnu/libedit/Makefile] Error 1
ym

[yassinm]

I am on ubuntu so i ended up doing installing libedit-dev from the system packaged

apt-get install autoconf bison build-essential libedit-dev

After that here is what i get

<snip> 

github.com/cockroachdb/cockroach/pkg/ccl/streamingccl/streamclient
github.com/cockroachdb/cockroach/pkg/migration/migrationmanager
github.com/cockroachdb/cockroach/pkg/ccl/streamingccl/streamproducer
github.com/cockroachdb/cockroach/pkg/ccl/streamingccl/streamingest
github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl
github.com/cockroachdb/cockroach/pkg/ccl/backupccl
github.com/cockroachdb/cockroach/pkg/ccl/gssapiccl
github.com/cockroachdb/cockroach/pkg/server
github.com/cockroachdb/cockroach/pkg/ccl/importccl
github.com/cockroachdb/cockroach/pkg/ccl/oidcccl
github.com/cockroachdb/cockroach/pkg/cli
github.com/cockroachdb/cockroach/pkg/ccl/cliccl
github.com/cockroachdb/cockroach/pkg/ccl
github.com/cockroachdb/cockroach/pkg/cmd/cockroach
go install -v docgen
bin/prereqs ./pkg/cmd/docgen > bin/docgen.d.tmp
mv -f bin/docgen.d.tmp bin/docgen.d
docgen grammar bnf docs/generated/sql/bnf --quiet
touch bin/.docgen_bnfs
docgen functions docs/generated/sql --quiet
touch bin/.docgen_functions
docgen http \
--protoc /home/ymo/3rdp/dev/golang/releases/golatest-packages/native/x86_64-linux-gnu/protobuf/protoc \
--gendoc ./bin/protoc-gen-doc \
--out docs/generated/http \
--protobuf pkg:./vendor/github.com:./vendor/github.com/gogo/protobuf:./vendor/github.com/gogo/protobuf/protobuf:./vendor/go.etcd.io:./vendor/github.com/grpc-ecosystem/grpc-gateway/third_party/googleapis:./vendor/github.com/cockroachdb/errors
touch bin/.docgen_http
docgen logformats docs/generated/logformats.md
touch bin/.docgen_logformats

Does this mean i am finally good ?

[dt]

If ./cockroach version prints out the current commit, you're all set.

[msirek]

It looks good. I see the line:
github.com/cockroachdb/cockroach/pkg/cmd/cockroach

If it was successful, re-running make should produce very minimal output this time, something like:

$ make build
Running make with -j32
GOPATH set to /home/msirek/go
make  all-am
make[2]: Nothing to be done for 'all-am'.
mkdir -p lib
./build/bazelutil/generate_redact_safe.sh >docs/generated/redact_safe.md.tmp || { rm -f docs/generated/redact_safe.md.tmp; exit 1; }
rm -f lib/lib{geos,geos_c}.so
cp -L /home/msirek/go/native/x86_64-linux-gnu/geos/lib/lib{geos,geos_c}.so lib
GOFLAGS= go build -o cockroach -v  -mod=vendor -gcflags "all=-N -l" -tags ' gss make x86_64_linux_gnu' -ldflags '-X github.com/cockroachdb/cockroach/pkg/build.typ=development -extldflags "" -X "github.com/cockroachdb/cockroach/pkg/build.tag=v22.1.0-alpha.00000000-2675-g3f078d4d89-dirty" -X "github.com/cockroachdb/cockroach/pkg/build.rev=3f078d4d8951872d898bdcc4d608122bdb6bac81" -X "github.com/cockroachdb/cockroach/pkg/build.cgoTargetTriple=x86_64-linux-gnu"   -X "github.com/cockroachdb/cockroach/pkg/build.utcTime=2022/01/14 12:57:14"' ./pkg/cmd/cockroach