Unable to build simple app image - following steps from blog

[jerrinsg]

Hello,

What steps did you take:

I was following along the steps mentioned in the "Deploying Kubernetes Applications with ytt, kbld, and kapp" blog.

What happened:

Under the "Building container images locally" section, the command to build the simple app image is listed as

$ kapp deploy -a simple-app -c -f <(ytt -f config-step-3-build-local/ -v hello_msg=friend | kbld -f-)

But I am seeing the following build error while running this command:

1:36:59PM: ongoing: reconcile deployment/simple-app (apps/v1) namespace: default
1:36:59PM:  ^ Waiting for 1 unavailable replicas
1:36:59PM:  L ok: waiting on replicaset/simple-app-7db87685bd (apps/v1) namespace: default
1:36:59PM:  L ok: waiting on replicaset/simple-app-78d7ff4f98 (apps/v1) namespace: default
1:36:59PM:  L ok: waiting on replicaset/simple-app-59f9fb6ff6 (apps/v1) namespace: default
1:36:59PM:  L ok: waiting on replicaset/simple-app-59b5c7ff8d (apps/v1) namespace: default
1:36:59PM:  L ongoing: waiting on pod/simple-app-7db87685bd-4h8p8 (v1) namespace: default
1:36:59PM:     ^ Pending: ErrImagePull (message: rpc error: code = Unknown desc = failed to pull and unpack image "docker.io/library/kbld:docker-io-dkalinin-k8s-simple-app-sha256-ebcc4c1710480262a12cd3d7caaa21bfe9d79a1688c6bc9fd2fffe7ef8d0017b": failed to resolve reference "docker.io/library/kbld:docker-io-dkalinin-k8s-simple-app-sha256-ebcc4c1710480262a12cd3d7caaa21bfe9d79a1688c6bc9fd2fffe7ef8d0017b": pull access denied, repository does not exist or may require authorization: server message: insufficient_scope: authorization failed)

See full logs here - deploy-err.txt

What did you expect:

Successful build of the app image as mentioned in the blog.

Anything else you would like to add:
None

Environment:

• kapp version (use kapp --version):

$ kapp --version
kapp version 0.45.0

• OS (e.g. from /etc/os-release):

$ sw_vers
ProductName:		macOS
ProductVersion:		13.0.1
BuildVersion:		22A400

• Kubernetes version (use kubectl version)

$ kubectl version
WARNING: This version information is deprecated and will be replaced with the output from kubectl version --short.  Use --output=yaml|json to get the full version.
Client Version: version.Info{Major:"1", Minor:"26", GitVersion:"v1.26.2", GitCommit:"fc04e732bb3e7198d2fa44efa5457c7c6f8c0f5b", GitTreeState:"clean", BuildDate:"2023-02-22T13:32:21Z", GoVersion:"go1.20.1", Compiler:"gc", Platform:"darwin/amd64"}
Kustomize Version: v4.5.7
Server Version: version.Info{Major:"1", Minor:"25", GitVersion:"v1.25.3", GitCommit:"434bfd82814af038ad94d62ebe59b133fcb50506", GitTreeState:"clean", BuildDate:"2022-10-25T19:35:11Z", GoVersion:"go1.19.2", Compiler:"gc", Platform:"linux/amd64"}

---

Vote on this request

This is an invitation to the community to vote on issues, to help us prioritize our backlog. Use the "smiley face" up to the right of this comment to vote.

👍 "I would like to see this addressed as soon as possible"
👎 "There are other more important things to focus on right now"

We are also happy to receive and review Pull Requests if you want to help working on this issue.

[praveenrewar]

Hi @jerrinsg I followed the steps from the blog and didn't face any issue. Can you make sure that you are logged in using your docker account?

[jerrinsg]

Thanks for the response @praveenrewar. Hmm I tried again after doing docker login and still see the same.

❯  docker login
Authenticating with existing credentials...
Login Succeeded

Logging in with your password grants your terminal complete access to your account.
For better security, log in with a limited-privilege personal access token. Learn more at https://docs.docker.com/go/access-tokens/

❯ kapp deploy -a simple-app -f config-step-1-minimal/
..
Succeeded

❯ kapp deploy -a simple-app -c -f <(ytt -f config-step-2-template/ -v hello_msg=friend)
..
Succeeded

❯  kapp deploy -a simple-app -c -f <(ytt -f config-step-3-build-local/ -v hello_msg=friend | kbld -f-)
...
4:52:43PM:     ^ Pending: ErrImagePull (message: rpc error: code = Unknown desc = failed to pull and unpack image "docker.io/library/kbld:docker-io-dkalinin-k8s-simple-app-sha256-9bb10ebf4d8d467ed9a625cdf86c1bd6be6a0ab308aaf80571aca36ca4c7c5fc": failed to resolve reference "docker.io/library/kbld:docker-io-dkalinin-k8s-simple-app-sha256-9bb10ebf4d8d467ed9a625cdf86c1bd6be6a0ab308aaf80571aca36ca4c7c5fc": pull access denied, repository does not exist or may require authorization: server message: insufficient_scope: authorization failed)
...
kapp: Error: waiting on reconcile deployment/simple-app (apps/v1) namespace: default:
  Finished unsuccessfully (Deployment is not progressing: ProgressDeadlineExceeded (message: ReplicaSet "simple-app-56fcfcb7dc" has timed out progressing.))

Full logs attached - kapp-err.log

[praveenrewar]

@jerrinsg Do you see this image when you do docker images? Are you able to inspect it docker image inspect kbld:docker-io-dkalinin-k8s-simple-app-sha256-9bb10ebf4d8d467ed9a625cdf86c1bd6be6a0ab308aaf80571aca36ca4c7c5fc?
I have faced a similar kind of issue once, and for me restarting my system seemed to have work. I wasn't able to debug what the issue was, but maybe you could give it a shot.

[jerrinsg]

Yes I see the image in docker images and can inspect it too:

$ docker images
REPOSITORY                                       TAG                                                                                                         IMAGE ID       CREATED         SIZE
kbld                                             docker-io-dkalinin-k8s-simple-app-sha256-9bb10ebf4d8d467ed9a625cdf86c1bd6be6a0ab308aaf80571aca36ca4c7c5fc   9bb10ebf4d8d   2 weeks ago     7.33MB

$ docker image inspect kbld:docker-io-dkalinin-k8s-simple-app-sha256-9bb10ebf4d8d467ed9a625cdf86c1bd6be6a0ab308aaf80571aca36ca4c7c5fc
[
    {
        "Id": "sha256:9bb10ebf4d8d467ed9a625cdf86c1bd6be6a0ab308aaf80571aca36ca4c7c5fc",
        "RepoTags": [
            "kbld:docker-io-dkalinin-k8s-simple-app-sha256-9bb10ebf4d8d467ed9a625cdf86c1bd6be6a0ab308aaf80571aca36ca4c7c5fc"
        ],
        "RepoDigests": [],
        "Parent": "",
        "Comment": "buildkit.dockerfile.v0",
        "Created": "2023-03-23T23:45:12.884589899Z",
        "Container": "",
        "ContainerConfig": {
            "Hostname": "",
            "Domainname": "",
            "User": "",
            "AttachStdin": false,
            "AttachStdout": false,
            "AttachStderr": false,
            "Tty": false,
            "OpenStdin": false,
            "StdinOnce": false,
            "Env": null,
            "Cmd": null,
            "Image": "",
            "Volumes": null,
            "WorkingDir": "",
            "Entrypoint": null,
            "OnBuild": null,
            "Labels": null
        },
        "DockerVersion": "",
        "Author": "",
        "Config": {
            "Hostname": "",
            "Domainname": "",
            "User": "",
            "AttachStdin": false,
            "AttachStdout": false,
            "AttachStderr": false,
            "ExposedPorts": {
                "80/tcp": {}
            },
            "Tty": false,
            "OpenStdin": false,
            "StdinOnce": false,
            "Env": [
                "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
            ],
            "Cmd": null,
            "Image": "",
            "Volumes": null,
            "WorkingDir": "/",
            "Entrypoint": [
                "/app"
            ],
            "OnBuild": null,
            "Labels": null
        },
        "Architecture": "amd64",
        "Os": "linux",
        "Size": 7328808,
        "VirtualSize": 7328808,
        "GraphDriver": {
            "Data": {
                "MergedDir": "/var/lib/docker/overlay2/j49tm5dtdhro8n8zx6dvm8wgz/merged",
                "UpperDir": "/var/lib/docker/overlay2/j49tm5dtdhro8n8zx6dvm8wgz/diff",
                "WorkDir": "/var/lib/docker/overlay2/j49tm5dtdhro8n8zx6dvm8wgz/work"
            },
            "Name": "overlay2"
        },
        "RootFS": {
            "Type": "layers",
            "Layers": [
                "sha256:b19800512795956b0202b1b1591009585193c46c68b318d35c7a3995ce2e78a5"
            ]
        },
        "Metadata": {
            "LastTagTime": "2023-04-07T22:36:02.856224926Z"
        }
    }
]

Restarting the system didn't help too :(