Accessing logs of a container started with AutoRemove: true

[qknight]

Description

Motivation

The CLI usage is alright and logs every event just as expected. However, when using the Go abstraction with ContainerCreate there seems no way to access the logged events leaving me with no way see what happened.

CLI usage

I want to run this command inside a container:

docker run -it --rm \
  --mount type=volume,src=synapse-data,dst=/data \
  -e SYNAPSE_SERVER_NAME=my.matrix.host \
  -e SYNAPSE_REPORT_STATS=yes \
  matrixdotorg/synapse:latest generate

Based on https://github.com/matrix-org/synapse/tree/v1.56.0/docker

Docker SDK usage

And I'm using this abstraction: https://pkg.go.dev/github.com/docker/docker/client#Client.ContainerCreate

As a general concept I want to use:

AutoRemove: true,

The point is to automate/enforce containers deletion after use, for instance, if the setup exits unexpectedly. I'm also using a container name: server_setup_temporary_container which hints the user that this is used during setup and is meant to be temporary. In case the setup did not shutdown the container, the user can do this and the bound volumes are freed.

My problem with this generate script

I can't use https://github.com/moby/moby/blob/v20.10.18/client/container_logs.go#L36 as the container exits once it finished executing the generate. Therefore I can't access the logs at all as they are already deleted.

In contrast, this works well with the postgresql container, as it runs as a daemon and needs explicit shutdown. The same concept fails with only executing a script!

I don't know how to continue here.

A few thoughts I had:

• after generate execute a 'sleep 3600' and then explicitly shut the container down as well
• try to get the logs from ContainerStart or ContainerCreate directly but studying the API this is probably not implemented this way

What I would not want is to remove the AutoRemove: true concept.

The source code

Using my StartContainer abstraction

// Start and run container
containerId, err := s.dockerClient.myStartContainer(docker.ContainerStartConfig{
	Image: matrixImage,
	Volumes: []docker.ContainerVolume{
		docker.ContainerVolume{
			Source: volume,
			Target: "/data",
		},
	},
	Env: []string{
		fmt.Sprintf("SYNAPSE_SERVER_NAME=%s", domain),
		"SYNAPSE_REPORT_STATS=no",
	},
	Cmds: []string{
		"generate",
	},
})

StartContainer abstraction

func (c *Client) myStartContainer(cfg ContainerStartConfig) (string, error) {
	if c.client == nil {
		return "", errors.New(noClientErr)
	}

	if len(cfg.Image) == 0 {
		return "", errors.New(noImageErr)
	}

	containerConfig := container.Config{
		Image: cfg.Image,
	}
	hostConfig := container.HostConfig{
		AutoRemove: true,
	}
	if cfg.Env != nil {
		containerConfig.Env = cfg.Env
	}
	if cfg.Cmds != nil {
		containerConfig.Cmd = make(strslice.StrSlice, len(cfg.Cmds))
		for i, _cmd := range cfg.Cmds {
			containerConfig.Cmd[i] = _cmd
		}
	}
	if cfg.Volumes != nil {
		hostConfig.Mounts = make([]mount.Mount, len(cfg.Volumes))
		for i, v := range cfg.Volumes {
			hostConfig.Mounts[i] = mount.Mount{
				Type:   "volume",
				Source: v.Source,
				Target: v.Target,
			}
		}
	}
	var networkingConfig *network.NetworkingConfig
	if cfg.Networks != nil {
		networkingConfig = &network.NetworkingConfig{EndpointsConfig: map[string]*network.EndpointSettings{}}
		for _, nw := range cfg.Networks {
			n := nw.Name
			networkingConfig.EndpointsConfig[n] = &network.EndpointSettings{Aliases: nw.Aliases}
		}
	}
	cont, err := c.client.ContainerCreate(
		c.ctx,
		&containerConfig,
		&hostConfig,
		networkingConfig,
		nil,
		"server_setup_temporary_container",
	)
	if err != nil {
		return "", err
	}
	colorlogger.Log.Info("Container ID of "+colorlogger.LYellow, cfg.Image, colorlogger.CLR+" is "+cont.ID)
	if err := c.client.ContainerStart(c.ctx, cont.ID, types.ContainerStartOptions{}); err != nil {
		return "", err
	}
	return cont.ID, nil
}

Greater scenario

I'm executing this setup in order to configure the containers which are later executed with 'docker compose' as some of the setups require explicit changes to the containers and can't be done declaratively.

StackOverflow thread

https://stackoverflow.com/questions/73731084/how-to-access-container-logs-of-a-script-executed-with-startcontainer-using-the