panic: runtime error: invalid memory address or nil pointer dereference

[skooter777]

Confirmation

• My issue isn't already found on the issue tracker.
• I have replicated my issue using the latest version of the library and it is still present.

cf-terraforming version

v0.18.0 and up

Expected outcome

I was expecting to receive:

terraform init && cf-terraforming generate --zone "ZONE" --resource-type "cloudflare_record" --email $TF_VAR_cf_mail --token $TF_VAR_cf_api_token -vvv || true

Initializing the backend...

Initializing provider plugins...

• Reusing previous version of cloudflare/cloudflare from the dependency lock file
• Using previously-installed cloudflare/cloudflare v4.30.0

Terraform has been successfully initialized!

You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.

If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.

DEBU[0000] initializing cloudflare-go with API Token account_Id= zone_id=******
DEBU[0004] initializing Terraform in .
DEBU[0004] reading Terraform schema for Cloudflare provider
DEBU[0004] beginning to read and build cloudflare_record resources
2024/04/26 17:30:50
GET /client/v4/zones/******/dns_records?page=1&per_page=100 HTTP/1.1
Host: api.cloudflare.com
User-Agent: cloudflare-go/v4
Authorization: Bearer [redacted]
Content-Type: application/json
Accept-Encoding: gzip

2024/04/26 17:30:51
HTTP/2.0 200 OK
Cf-Cache-Status: DYNAMIC
Cf-Ray: 87a84d9678c337ca-FRA
Content-Type: application/json
Date: Fri, 26 Apr 2024 17:30:52 GMT
Server: cloudflare
Set-Cookie: __cflb=0H28vgHxwvgAQt********TKFwHfQtFKK; SameSite=Lax; path=/; expires=Fri, 26-Apr-24 20:00:53 GMT; HttpOnly
Set-Cookie: __cfruid=3c7f53b59122*******2ca64fb3343b4a-1714152652; path=/; domain=.api.cloudflare.com; HttpOnly; Secure; SameSite=None
Vary: Accept-Encoding

{"result":[{"id":"d8e0f3cc452c8fc6636ab60","zone_id":"","zone_name":"domain*****.link","name":"www.domain******.link","type":"AAAA","content":"100::","proxiable":true,"proxied":true,"ttl":1,"locked":false,"meta":{"auto_added":false,"managed_by_apps":false,"managed_by_argo_tunnel":false},"comment":null,"tags":[],"created_on":"2024-04-26T12:20:40.237697Z","modified_on":"2024-04-26T12:20:40.237697Z"},{"id":"236747321ebf18081a84709086c616d1","zone_id":"","zone_name":"domain.link","name":"_acme-challenge.domain******.link","type":"CNAME","content":"domain******.link.4f882c14279cc60d.dcv.cloudflare.com","proxiable":true,"proxied":false,"ttl":60,"locked":false,"meta":{"auto_added":false,"managed_by_apps":false,"managed_by_argo_tunnel":false},"comment":null,"tags":[],"created_on":"2024-04-26T12:20:38.052793Z","modified_on":"2024-04-26T12:20:38.052793Z"},{"id":"60e91648764cf04e3f12497e","zone_id":"","zone_name":"domain******.link","name":"domain******.link","type":"CNAME","content":"domain..m.xyz","proxiable":true,"proxied":false,"ttl":60,"locked":false,"meta":{"auto_added":false,"managed_by_apps":false,"managed_by_argo_tunnel":false},"comment":null,"tags":[],"created_on":"2024-04-26T12:20:39.015434Z","modified_on":"2024-04-26T12:20:39.015434Z"},{"id":"a65e8f6d5a61d3f4a6b3f05","zone_id":"","zone_name":"domain*****.link","name":"m.domain******.link","type":"CNAME","content":"domain******.link","proxiable":true,"proxied":true,"ttl":1,"locked":false,"meta":{"auto_added":false,"managed_by_apps":false,"managed_by_argo_tunnel":false},"comment":null,"tags":[],"created_on":"2024-04-26T12:20:38.777283Z","modified_on":"2024-04-26T12:20:38.777283Z"}],"success":true,"errors":[],"messages":[],"result_info":{"page":1,"per_page":100,"count":4,"total_count":4,"total_pages":1}}
DEBU[0005] got unknown attribute configuration: key allow_overwrite, value , value type
DEBU[0005] got unknown attribute configuration: key comment, value , value type
DEBU[0005] got unknown attribute configuration: key priority, value , value type
DEBU[0005] got unknown attribute configuration: key tags, value , value type
DEBU[0005] got unknown attribute configuration: key allow_overwrite, value , value type
DEBU[0005] got unknown attribute configuration: key comment, value , value type
DEBU[0005] got unknown attribute configuration: key priority, value , value type
DEBU[0005] got unknown attribute configuration: key tags, value , value type
DEBU[0005] got unknown attribute configuration: key allow_overwrite, value , value type
DEBU[0005] got unknown attribute configuration: key comment, value , value type
DEBU[0005] got unknown attribute configuration: key priority, value , value type
DEBU[0005] got unknown attribute configuration: key tags, value , value type
DEBU[0005] got unknown attribute configuration: key allow_overwrite, value , value type
DEBU[0005] got unknown attribute configuration: key comment, value , value type
DEBU[0005] got unknown attribute configuration: key priority, value , value type
DEBU[0005] got unknown attribute configuration: key tags, value , value type
resource "cloudflare_record" "terraform_managed_resource_d8e0f3cc452cab98fc6636ab60" {
name = "www"
proxied = true
ttl = 1
type = "AAAA"
value = "100::"
zone_id = "*"
}

resource "cloudflare_record" "terraform_managed_resource_236747321ebf09086c616d1" {
name = "_acme-challenge"
proxied = false
ttl = 60
type = "CNAME"
value = "domain.link.4f8******.cloudflare.com"
zone_id = "******"
}

Actual outcome

terraform init && cf-terraforming generate --zone "ZONE" --resource-type "cloudflare_record" --email $TF_VAR_cf_mail --token $TF_VAR_cf_api_token -vvv || true

Initializing the backend...
Initializing provider plugins...

• Finding cloudflare/cloudflare versions matching "~> 4.0"...
• Installing cloudflare/cloudflare v4.30.0...
• Installed cloudflare/cloudflare v4.30.0 (self-signed, key ID C76001609EE3B136)
  Partner and community providers are signed by their developers.
  If you'd like to know more about provider signing, you can read about it here:
  https://www.terraform.io/docs/cli/plugins/signing.html
  Terraform has created a lock file .terraform.lock.hcl to record the provider
  selections it made above. Include this file in your version control repository
  so that Terraform can guarantee to make the same selections by default when
  you run "terraform init" in the future.
  Terraform has been successfully initialized!
  You may now begin working with Terraform. Try running "terraform plan" to see
  any changes that are required for your infrastructure. All Terraform commands
  should now work.
  If you ever set or change modules or backend configuration for Terraform,
  rerun this command to reinitialize your working directory. If you forget, other
  commands will detect it and remind you to do so if necessary.

time="2024-04-26T17:16:40Z" level=debug msg="initializing cloudflare-go with API Token" account_Id= zone_id=ZONE
time="2024-04-26T17:16:41Z" level=debug msg="initializing Terraform in ."
time="2024-04-26T17:16:41Z" level=debug msg="reading Terraform schema for Cloudflare provider"
time="2024-04-26T17:16:42Z" level=debug msg="beginning to read and build cloudflare_record resources"
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x70 pc=0x79981e]
goroutine 1 [running]:
github.com/cloudflare/cloudflare-go.(*API).makeRequestContext(0xc000366040?, {0xc80a50?, 0xc000126000?}, {0xb6d795?, 0x1?}, {0xc00036ae00?, 0x470ee5?}, {0x0?, 0x0?})
/home/runner/go/pkg/mod/github.com/cloudflare/cloudflare-go@v0.85.0/cloudflare.go:172 +0x3e
github.com/cloudflare/cloudflare-go.(*API).ListDNSRecords(, {, _}, _, {{0x0, 0x0}, {0x0, 0x0}, {0x0, 0x0}, ...})
/home/runner/go/pkg/mod/github.com/cloudflare/cloudflare-go@v0.85.0/dns.go:252 +0x21a
github.com/cloudflare/cf-terraforming/internal/app/cf-terraforming/cmd.generateResources.func1(0x1088660?, {0xb6dc07?, 0x9?, 0x9?})
/home/runner/work/cf-terraforming/cf-terraforming/internal/app/cf-terraforming/cmd/generate.go:703 +0x10cf
github.com/spf13/cobra.(*Command).execute(0x1088660, {0xc000272360, 0x9, 0x9})
/home/runner/go/pkg/mod/github.com/spf13/cobra@v1.8.0/command.go:987 +0xa91
github.com/spf13/cobra.(*Command).ExecuteC(0x1088380)
/home/runner/go/pkg/mod/github.com/spf13/cobra@v1.8.0/command.go:1115 +0x425
github.com/spf13/cobra.(*Command).Execute(...)
/home/runner/go/pkg/mod/github.com/spf13/cobra@v1.8.0/command.go:1039
github.com/cloudflare/cf-terraforming/internal/app/cf-terraforming/cmd.Execute()
/home/runner/work/cf-terraforming/cf-terraforming/internal/app/cf-terraforming/cmd/root.go:30 +0x25
main.main()
/home/runner/work/cf-terraforming/cf-terraforming/cmd/cf-terraforming/main.go:8 +0x17

Steps to reproduce

Dockerfile latest version

#ARG GOARCH=arm64
ARG GOARCH=amd64

FROM golang:1.21.3-bullseye as build
RUN git clone https://github.com/cloudflare/cf-terraforming.git /go/cf-terraforming &&
cd /go/cf-terraforming &&
CGO_ENABLED=0 GOARCH=${GOARCH} go build -o cf-terraforming cmd/cf-terraforming/main.go

FROM python:3.6
COPY --from=build /go/cf-terraforming/cf-terraforming /usr/local/bin/cf-terraforming

RUN apt update &&
apt install -y lsb-release wget &&
wget -O- https://apt.releases.hashicorp.com/gpg | gpg --dearmor -o /usr/share/keyrings/hashicorp-archive-keyring.gpg &&
echo "deb [signed-by=/usr/share/keyrings/hashicorp-archive-keyring.gpg] https://apt.releases.hashicorp.com $(lsb_release -cs) main" | tee /etc/apt/sources.list.d/hashicorp.list &&
apt update && apt install -y terraform &&
apt clean && rm -rf /var/lib/apt/lists/*

RUN pip install --no-cache-dir requests pyyaml
WORKDIR /app

And I also tested versions lower than the latest one.

FROM debian:bullseye-slim as build
RUN apt-get update && apt-get install -y wget tar &&
wget https://github.com/cloudflare/cf-terraforming/releases/download/v0.18.0/cf-terraforming_0.18.0_linux_amd64.tar.gz &&
tar -xzf cf-terraforming_0.18.0_linux_amd64.tar.gz -C /usr/local/bin

FROM python:3.6
COPY --from=build /usr/local/bin/cf-terraforming /usr/local/bin/cf-terraforming

RUN apt update &&
apt install -y lsb-release wget &&
wget -O- https://apt.releases.hashicorp.com/gpg | gpg --dearmor -o /usr/share/keyrings/hashicorp-archive-keyring.gpg &&
echo "deb [signed-by=/usr/share/keyrings/hashicorp-archive-keyring.gpg] https://apt.releases.hashicorp.com $(lsb_release -cs) main" | tee /etc/apt/sources.list.d/hashicorp.list &&
apt update && apt install -y terraform &&
apt clean && rm -rf /var/lib/apt/lists/*
RUN pip install --no-cache-dir requests pyyaml

As a result, containers built on a Mac (M1) work as expected, but those built on the amd64 architecture using Kaniko return an error.

References

No response

[skooter777]

I've tried various versions of both Golang and build systems, as well as different versions of the application itself, but the error remains the same.

[BogdanSorlea]

I have the same (or similar) problem, running it in Github actions with

go version go1.22.3 linux/amd64
time="2024-05-10T07:02:08Z" level=error msg="failed to exec to `git`"
cf-terraforming -dev+a102dba56329

golang installed based on https://github.com/actions/setup-go
no amount of pinning versions around (golang version, cf-terraforming version) makes any difference
errors I get are

c582740762f62fb2625812a2a8c54687 - cloudflare_access_application...
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x70 pc=0x79bf78]
goroutine 1 [running]:
github.com/cloudflare/cloudflare-go.(*API).makeRequestContext(0xc000[33](https://github.com/worksome/terraform/actions/runs/9028553445/job/24809247021#step:7:34)5580?, {0xc88210?, 0x11373c0?}, {0xb6a9fc?, 0x2?}, {0xc00013d280?, 0x0?}, {0x0?, 0x0?})
	/home/runner/go/pkg/mod/github.com/cloudflare/cloudflare-go@v0.90.0/cloudflare.go:172 +0x38
github.com/cloudflare/cloudflare-go.(*API).ListAccessApplications(0x0, {0xc88210, 0x11373c0}, 0xc0005467b0, {{0x1, 0x19, 0x0, 0x0, 0x0, {0x0, ...}, ...}})
	/home/runner/go/pkg/mod/github.com/cloudflare/cloudflare-go@v0.90.0/access_application.go:247 +0x1cf
github.com/cloudflare/cf-terraforming/internal/app/cf-terraforming/cmd.init.generateResources.func3(0x10ce[38](https://github.com/worksome/terraform/actions/runs/9028553445/job/24809247021#step:7:39)0, {0xb6aca9?, 0x4?, 0xb6ac2d?})
	/home/runner/go/pkg/mod/github.com/cloudflare/cf-terraforming@v0.19.0/internal/app/cf-terraforming/cmd/generate.go:114 +0x3daf
github.com/spf13/cobra.(*Command).execute(0x10ce380, {0xc0001084e0, 0x6, 0x6})
	/home/runner/go/pkg/mod/github.com/spf13/cobra@v1.8.0/command.go:987 +0xab1
c582740762f62fb2625812a2a8c54687 - cloudflare_access_ca_certificate...
github.com/spf13/cobra.(*Command).ExecuteC(0x10ce0a0)
	/home/runner/go/pkg/mod/github.com/spf13/cobra@v1.8.0/command.go:1115 +0x3ff
github.com/spf13/cobra.(*Command).Execute(...)
	/home/runner/go/pkg/mod/github.com/spf13/cobra@v1.8.0/command.go:10[39](https://github.com/worksome/terraform/actions/runs/9028553445/job/24809247021#step:7:40)
github.com/cloudflare/cf-terraforming/internal/app/cf-terraforming/cmd.Execute()
	/home/runner/go/pkg/mod/github.com/cloudflare/cf-terraforming@v0.19.0/internal/app/cf-terraforming/cmd/root.go:30 +0x1a
main.main()
	/home/runner/go/pkg/mod/github.com/cloudflare/cf-terraforming@v0.19.0/cmd/cf-terraforming/main.go:8 +0xf

command I run is along the lines of

cf-terraforming generate -t $CLOUDFLARE_READONLY_API_TOKEN --resource-type "$resourcetype" --zone $zone > $outfolder/$zone.$resourcetype.tf

[BogdanSorlea]

With the help of the Golang community granting me a second eye, we were able to find the likely culprit in the CI=true env var that Github Actions sets by default, so the immediate workaround for me is to just unset it (at least during the run of this cf-terraforming command).

I would recommend though that maybe the maintainers of the repo could consider a different strategy for what they are trying to achieve, one that could range from using a different env var to at least documenting it better (currently missing from docs, as far as I could tell).