Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

nodes metadata JSON ouput #162

Merged
merged 4 commits into from Sep 22, 2022
Merged

nodes metadata JSON ouput #162

merged 4 commits into from Sep 22, 2022

Conversation

crazy-max
Copy link
Member

@crazy-max crazy-max commented Sep 18, 2022

At the moment endpoint, status, flags and platforms outputs are only return for the very first note being registered for the builder.

This PR adds a new output nodes that returns metadata JSON of all registered nodes and also adds new fields available since Buildx 0.9:

[
  {
    "name": "builder-06133552-7a7d-4d4f-a011-fccc08c3ae500",
    "endpoint": "unix:///var/run/docker.sock",
    "driver-opts": [
      "image=moby/buildkit:master",
      "network=host"
    ],
    "status": "running",
    "buildkitd-flags": "--allow-insecure-entitlement security.insecure --allow-insecure-entitlement network.host",
    "buildkit": "81b6ff2",
    "platforms": "linux/amd64,linux/amd64/v2,linux/amd64/v3,linux/386"
  }
]

This change is primarily for future support of appending nodes to a builder, so the output is relevant.

endpoint, status, flags and platforms outputs are deprecated and will be removed in next major release.

Copy link
Member

@tonistiigi tonistiigi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I could maybe see status and platforms being a little bit useful in some cases. If the goal is to handle the cases with multiple nodes(not very common in CI) then maybe we could just combine the values for these fields?

Copy link
Contributor

@jedevc jedevc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I could maybe see status and platforms being a little bit useful in some cases. If the goal is to handle the cases with multiple nodes(not very common in CI) then maybe we could just combine the values for these fields?

Agreed, especially makes sense for platforms, I like the idea of merging each node's together for this if we can.

src/main.ts Show resolved Hide resolved
README.md Outdated Show resolved Hide resolved
@crazy-max
Copy link
Member Author

crazy-max commented Sep 20, 2022

I could maybe see status and platforms being a little bit useful in some cases. If the goal is to handle the cases with multiple nodes(not very common in CI) then maybe we could just combine the values for these fields?

Agreed, especially makes sense for platforms, I like the idea of merging each node's together for this if we can.

This would not give the expected result as we can define preferred platforms even if those are not supported on specified nodes:

  {
    "nodes": [
      {
        "name": "builder-13dbff8e-3be6-4024-9aa5-2c75782f1cf10",
        "endpoint": "docker-container://buildx_buildkit_builder-ba6a88cb-fe10-4412-876e-66458fe474180",
        "status": "running",
        "platforms": "linux/amd64,linux/amd64/v2,linux/amd64/v3,linux/amd64/v4,linux/arm64,linux/riscv64,linux/ppc64le,linux/s390x,linux/386,linux/mips64le,linux/mips64,linux/arm/v7,linux/arm/v6"
      },
      {
        "name": "aws_graviton2",
        "endpoint": "tcp://***:1234",
        "driver-opts": [
          "cacert=/home/runner/.docker/buildx/creds/builder-13dbff8e-3be6-4024-9aa5-2c75782f1cf1/cacert_***-1234.pem",
          "cert=/home/runner/.docker/buildx/creds/builder-13dbff8e-3be6-4024-9aa5-2c75782f1cf1/cert_***-1234.pem",
          "key=/home/runner/.docker/buildx/creds/builder-13dbff8e-3be6-4024-9aa5-2c75782f1cf1/key_***-1234.pem"
        ],
        "status": "running",
        "platforms": "darwin/arm64*,linux/arm64*,linux/arm/v5*,linux/arm/v6*,linux/arm/v7*,windows/arm64*"
      },
      {
        "name": "linuxone_s390x",
        "endpoint": "tcp://***:1234",
        "driver-opts": [
          "cacert=/home/runner/.docker/buildx/creds/builder-13dbff8e-3be6-4024-9aa5-2c75782f1cf1/cacert_***-1234.pem",
          "cert=/home/runner/.docker/buildx/creds/builder-13dbff8e-3be6-4024-9aa5-2c75782f1cf1/cert_***-1234.pem",
          "key=/home/runner/.docker/buildx/creds/builder-13dbff8e-3be6-4024-9aa5-2c75782f1cf1/key_***-1234.pem"
        ],
        "status": "running",
        "platforms": "linux/s390x*"
      }
    ],
    "name": "builder-13dbff8e-3be6-4024-9aa5-2c75782f1cf1",
    "driver": "remote"
  }

For example here aws_graviton2 supported platforms are:

time="2022-09-19T21:12:20Z" level=info msg="found worker \"klrqif3uuxy84t69eo3eef89n\",
  labels=map[org.mobyproject.buildkit.worker.executor:oci
    org.mobyproject.buildkit.worker.hostname:7c099955516d
    org.mobyproject.buildkit.worker.network:host
    org.mobyproject.buildkit.worker.oci.process-mode:sandbox
    org.mobyproject.buildkit.worker.snapshotter:overlayfs
  ],
  platforms=[linux/arm64 linux/arm/v7 linux/arm/v6]"

But I define the following ones as preferred:

      -
        name: Set up Docker Buildx
        uses: crazy-max/docker-setup-buildx-action@append
        with:
          driver: remote
          endpoint: docker-container://buildx_buildkit_${{ steps.builder.outputs.name }}0
          append: |
            - name: aws_graviton2
              endpoint: tcp://${{ secrets.AWS_ARM64_HOST }}:1234
              platforms: darwin/arm64,linux/arm64,linux/arm/v5,linux/arm/v6,linux/arm/v7,windows/arm64
            - name: linuxone_s390x
              endpoint: tcp://${{ secrets.LINUXONE_S390X_HOST }}:1234
              platforms: linux/s390x

See #165 and https://github.com/crazy-max/docker-setup-buildx-action/blob/append/docs/advanced/append-nodes.md about the append input

@crazy-max
Copy link
Member Author

As discussed keep platforms output but return preferred ones if set or all available.

Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
@crazy-max crazy-max merged commit d5234d6 into docker:master Sep 22, 2022
@crazy-max crazy-max deleted the nodes-output branch September 22, 2022 09:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants