Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

support a docker flavor meant to be used as base image for container-based CI jobs #2325

Open
osher opened this issue Nov 21, 2022 · 6 comments · May be fixed by #2317
Open

support a docker flavor meant to be used as base image for container-based CI jobs #2325

osher opened this issue Nov 21, 2022 · 6 comments · May be fixed by #2317
Labels
area/ci-cd area/container For all bugs related to the kaniko container kind/feature-request priority/p2 High impact feature/bug. Will get a lot of users happy

Comments

@osher
Copy link

osher commented Nov 21, 2022

kaniko is used a lot as a base image for CI job-containers.
For example - Gitlab-CI instructs its users to use it. However, unintuitively, it requires:

  1. to use the kaniko:debug image - which feels like a hack
  2. and to override the entrypoint - also feels like a hack
  3. it comes with just /busybox, and is therefore very limiting when your job needs to do anything else before or after the build, like posting some metadata with curl.

I propose to support an image flavor kaniko:ci that is dedicated to container-based CI jobs, with:

  1. name kaniko:ci
  2. entrypoint bin/sh
  3. with a place for a utility pack for CI jobs, starting with curl and jq
@kumprj
Copy link

kumprj commented Nov 29, 2022

Is there a reason you need Kaniko to do anything other than build docker images? It should not be your CI tool, IMO.

We used Jenkins and used side-car containers. We had any docker-related building use the kaniko container, and our Jenkins Docker Image Build agent handle anything else. These shared a PVC so that files could be accessed by both.

It worked well. Is something like this possible with Gitlab-CI?

@osher
Copy link
Author

osher commented Jan 26, 2023

Well, there are sidecars, and there are services, but it's all very combersome and not reliable enough to my opinion.

We use Kaniko as a base image for a job, it runs the CLI, it produces the image and pushes it to a repo - that's all great.

But I need the same job to report the image somewhere using curl. Really? we need a side-car for that?
I don't want another job just for that!!
Besides - this curl report should be more or less atomic, esp. when we update moving tags like latest.
Another job is kinda klunky and fragile, and I have to use artifacts between them - really?

I'll give you more details: the job calculates the tag, and gathers build info into a json file. So I have to gather it as an artifact that is uploaded to S3, so the job with the curl can download it, just so it can post it to consul? all that because kaniko comes with plain busybox and nothing more?

We solved it by creating an image that extends that kaniko with busybox and adds to that curl and jq, and it works for us very well - so great! So I thought not to keep that to myself and created this #2317

@osher
Copy link
Author

osher commented Jan 26, 2023

I'll give you two more cents here.

Because of this limitation, gitlab moved to recommending launching kaniko from docker-in-docker, so you can have your base image with your tools and kaniko is lunched from docker-in-docker....? really? doesn't that defeat the purpose?

@gabyx
Copy link
Contributor

gabyx commented Feb 4, 2023

I did it differently: I prepared with python scripts a kaniko context where everything is resolved, it contains a Dockerfile contains maybe stupid credentials I need to mount in, contains a run.sh script which basically just calls the executor with all flags etc. In that way I can mount this folder into the Kaniko:debug Container in Jenkins and execute the script. So every postprocessing before calling the executor is done beforehand. This has the advantage not to do to much (impossible becuase the tools are not in the kaniko image) unrelated things in the kaniko image itself. So yes we do that in side-car containers to produce this context directory.

@aaron-prindle aaron-prindle added area/container For all bugs related to the kaniko container kind/feature-request priority/p2 High impact feature/bug. Will get a lot of users happy area/ci-cd labels Jun 25, 2023
@osher
Copy link
Author

osher commented Jul 3, 2023

@gabyx so you use docker-x or docker-in-docker to launch a kaniko container?

@gabyx
Copy link
Contributor

gabyx commented Jul 15, 2023

@osher: We use kubernetes over Jenkins CI to launch kaniko to build images. To launch a kaniko container you can use docker or podman, kaniko builds a container in OCI format, thats all it does. I did not understand the question probably?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/ci-cd area/container For all bugs related to the kaniko container kind/feature-request priority/p2 High impact feature/bug. Will get a lot of users happy
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants