feat: add clickhouse module (#1372)

* feat: add clickhouse module

* refactor minor bugs

* refactor tests

* minor changes

* Update docs/modules/clickhouse.md

Co-authored-by: Manuel de la Peña <social.mdelapenya@gmail.com>

* Update docs/modules/clickhouse.md

Co-authored-by: Manuel de la Peña <social.mdelapenya@gmail.com>

* use new version (v0.22.0) in clickhouse

* Add clickhouse to new workflow

* Refactor tests

* Remove unnecessary default env

---------

Co-authored-by: Manuel de la Peña <social.mdelapenya@gmail.com>
Co-authored-by: Manuel de la Peña <mdelapenya@gmail.com>

.github/dependabot.yml

@@ -106,6 +106,13 @@ updates:
         day: sunday
       open-pull-requests-limit: 3
       rebase-strategy: disabled
+    - package-ecosystem: gomod
+      directory: /modules/clickhouse
+      schedule:
+        interval: monthly
+        day: sunday
+      open-pull-requests-limit: 3
+      rebase-strategy: disabled
     - package-ecosystem: gomod
       directory: /modules/compose
       schedule:

.github/workflows/ci.yml

@@ -82,7 +82,7 @@ jobs:
       matrix:
         go-version: [1.19.x, 1.x]
         platform: [ubuntu-latest, macos-latest]
-        module: [compose, couchbase, k3s, localstack, mysql, neo4j, postgres, pulsar, redis, redpanda, vault]
+        module: [clickhouse, compose, couchbase, k3s, localstack, mysql, neo4j, postgres, pulsar, redis, redpanda, vault]
         exclude:
           - module: compose
             go-version: 1.19.x

docs/modules/clickhouse.md

@@ -0,0 +1,125 @@
+# ClickHouse
+
+Not available until the next release of testcontainers-go <a href="https://github.com/testcontainers/testcontainers-go"><span class="tc-version">:material-tag: main</span></a>
+
+## Introduction
+
+The Testcontainers module for ClickHouse.
+
+## Adding this module to your project dependencies
+
+Please run the following command to add the ClickHouse module to your Go dependencies:
+
+```
+go get github.com/testcontainers/testcontainers-go/modules/clickhouse
+```
+
+## Usage example
+
+<!--codeinclude-->
+
+[Test for a ClickHouse container](../../modules/clickhouse/clickhouse_test.go)inside_block:customInitialization
+
+<!--/codeinclude-->
+
+## Module reference
+
+The ClickHouse module exposes one entrypoint function to create the ClickHouse container, and this function receives two parameters:
+
+```golang
+func RunContainer(ctx context.Context, opts ...testcontainers.ContainerCustomizer) (*ClickHouseContainer, error)
+```
+
+- `context.Context`, the Go context.
+- `testcontainers.ContainerCustomizer`, a variadic argument for passing options.
+
+### Container Options
+
+When starting the ClickHouse container, you can pass options in a variadic way to configure it.
+
+#### Image
+
+If you need to set a different ClickHouse Docker image, you can use `testcontainers.WithImage` with a valid Docker image
+for ClickHouse. E.g. `testcontainers.WithImage("clickhouse/clickhouse-server:23.3.8.21-alpine")`.
+
+#### Wait Strategies
+
+If you need to set a different wait strategy for ClickHouse, you can use `testcontainers.WithWaitStrategy` with a valid wait strategy
+for ClickHouse.
+
+!!!info
+    The default deadline for the wait strategy is 60 seconds.
+
+At the same time, it's possible to set a wait strategy and a custom deadline with `testcontainers.WithWaitStrategyAndDeadline`.
+
+#### Docker type modifiers
+
+If you need an advanced configuration for ClickHouse, you can leverage the following Docker type modifiers:
+
+- `testcontainers.WithConfigModifier`
+- `testcontainers.WithHostConfigModifier`
+- `testcontainers.WithEndpointSettingsModifier`
+
+Please read the [Create containers: Advanced Settings](../features/creating_container.md#advanced-settings) documentation for more information.
+
+#### Set username, password and database name
+
+If you need to set a different database, and its credentials, you can use `WithUsername`, `WithPassword`, `WithDatabase`
+options.
+
+<!--codeinclude-->
+
+[Custom Database initialization](../../modules/clickhouse/clickhouse_test.go) inside_block:customInitialization
+
+<!--/codeinclude-->
+
+!!!info
+    The default values for the username is `default`, for password is `clickhouse` and for the default database name is `clickhouse`.
+
+#### Init Scripts
+
+If you would like to do additional initialization in the ClickHouse container, add one or more `*.sql`, `*.sql.gz`, or `*.sh` scripts to the container request.
+Those files will be copied after the container is created but before it's started under `/docker-entrypoint-initdb.d`. According to ClickHouse Docker image,
+it will run any `*.sql` files, run any executable `*.sh` scripts, and source any non-executable `*.sh` scripts found in that directory to do further
+initialization before starting the service.
+
+<!--codeinclude-->
+
+[Include init scripts](../../modules/clickhouse/clickhouse_test.go) inside_block:withInitScripts
+
+<!--/codeinclude-->
+
+<!--codeinclude-->
+
+[Init script content](../../modules/clickhouse/testdata/init-db.sh)
+
+<!--/codeinclude-->
+
+### Container Methods
+
+The ClickHouse container exposes the following methods:
+
+#### ConnectionString
+
+This method returns the dsn connection string to connect to the ClickHouse container, using the default configs.
+It's possible to pass extra parameters to the connection string, e.g. `dial_timeout=300ms` or `skip_verify=false`, in a variadic way.
+
+e.g. `clickhouse://default:pass@localhost:9000?dial_timeout=300ms&skip_verify=false`
+
+<!--codeinclude-->
+
+[Get connection string](../../modules/clickhouse/clickhouse_test.go) inside_block:connectionString
+
+<!--/codeinclude-->
+
+#### ConnectionHost
+
+This method returns the host & port of the ClickHouse container.
+
+e.g. `localhost:9000`
+
+<!--codeinclude-->
+
+[Get connection host](../../modules/clickhouse/clickhouse_test.go) inside_block:connectionHost
+
+<!--/codeinclude-->

mkdocs.yml

@@ -60,6 +60,7 @@ nav:
             - SQL: features/wait/sql.md
     - Modules:
         - modules/index.md
+        - modules/clickhouse.md
         - modules/couchbase.md
         - modules/k3s.md
         - modules/localstack.md

modules/clickhouse/Makefile

@@ -0,0 +1,5 @@
+include ../../commons-test.mk
+
+.PHONY: test
+test:
+	$(MAKE) test-clickhouse

modules/clickhouse/clickhouse.go

@@ -0,0 +1,178 @@
+package clickhouse
+
+import (
+	"context"
+	"fmt"
+	"path/filepath"
+	"strings"
+
+	"github.com/docker/go-connections/nat"
+	"github.com/testcontainers/testcontainers-go"
+	"github.com/testcontainers/testcontainers-go/wait"
+)
+
+const defaultUser = "default"
+const defaultDatabaseName = "clickhouse"
+
+const defaultImage = "clickhouse/clickhouse-server:23.3.8.21-alpine"
+
+const httpPort = nat.Port("8123/tcp")
+const nativePort = nat.Port("9000/tcp")
+
+// ClickHouseContainer represents the ClickHouse container type used in the module
+type ClickHouseContainer struct {
+	testcontainers.Container
+	dbName   string
+	user     string
+	password string
+}
+
+func (c *ClickHouseContainer) ConnectionHost(ctx context.Context) (string, error) {
+	host, err := c.Host(ctx)
+	if err != nil {
+		return "", err
+	}
+
+	port, err := c.MappedPort(ctx, nativePort)
+	if err != nil {
+		return "", err
+	}
+
+	return host + ":" + port.Port(), nil
+}
+
+// ConnectionString returns the dsn string for the clickhouse container, using the default 9000 port, and
+// obtaining the host and exposed port from the container. It also accepts a variadic list of extra arguments
+// which will be appended to the dsn string. The format of the extra arguments is the same as the
+// connection string format, e.g. "dial_timeout=300ms" or "skip_verify=false"
+func (c *ClickHouseContainer) ConnectionString(ctx context.Context, args ...string) (string, error) {
+	host, err := c.ConnectionHost(ctx)
+	if err != nil {
+		return "", err
+	}
+
+	extraArgs := ""
+	if len(args) > 0 {
+		extraArgs = strings.Join(args, "&")
+	}
+	if extraArgs != "" {
+		extraArgs = "?" + extraArgs
+	}
+
+	connectionString := fmt.Sprintf("clickhouse://%s:%s@%s/%s%s", c.user, c.password, host, c.dbName, extraArgs)
+	return connectionString, nil
+}
+
+// WithInitScripts sets the init scripts to be run when the container starts
+func WithInitScripts(scripts ...string) testcontainers.CustomizeRequestOption {
+	return func(req *testcontainers.GenericContainerRequest) {
+		initScripts := []testcontainers.ContainerFile{}
+		for _, script := range scripts {
+			cf := testcontainers.ContainerFile{
+				HostFilePath:      script,
+				ContainerFilePath: "/docker-entrypoint-initdb.d/" + filepath.Base(script),
+				FileMode:          0755,
+			}
+			initScripts = append(initScripts, cf)
+		}
+		req.Files = append(req.Files, initScripts...)
+	}
+}
+
+// WithConfigFile sets the XML config file to be used for the clickhouse container
+// It will also set the "configFile" parameter to the path of the config file
+// as a command line argument to the container.
+func WithConfigFile(configFile string) testcontainers.CustomizeRequestOption {
+	return func(req *testcontainers.GenericContainerRequest) {
+		cf := testcontainers.ContainerFile{
+			HostFilePath:      configFile,
+			ContainerFilePath: "/etc/clickhouse-server/config.d/config.xml",
+			FileMode:          0755,
+		}
+		req.Files = append(req.Files, cf)
+	}
+}
+
+// WithConfigFile sets the YAML config file to be used for the clickhouse container
+// It will also set the "configFile" parameter to the path of the config file
+// as a command line argument to the container.
+func WithYamlConfigFile(configFile string) testcontainers.CustomizeRequestOption {
+	return func(req *testcontainers.GenericContainerRequest) {
+		cf := testcontainers.ContainerFile{
+			HostFilePath:      configFile,
+			ContainerFilePath: "/etc/clickhouse-server/config.d/config.yaml",
+			FileMode:          0755,
+		}
+		req.Files = append(req.Files, cf)
+	}
+}
+
+// WithDatabase sets the initial database to be created when the container starts
+// It can be used to define a different name for the default database that is created when the image is first started.
+// If it is not specified, then the default value("clickhouse") will be used.
+func WithDatabase(dbName string) testcontainers.CustomizeRequestOption {
+	return func(req *testcontainers.GenericContainerRequest) {
+		req.Env["CLICKHOUSE_DB"] = dbName
+	}
+}
+
+// WithPassword sets the initial password of the user to be created when the container starts
+// It is required for you to use the ClickHouse image. It must not be empty or undefined.
+// This environment variable sets the password for ClickHouse.
+func WithPassword(password string) testcontainers.CustomizeRequestOption {
+	return func(req *testcontainers.GenericContainerRequest) {
+		req.Env["CLICKHOUSE_PASSWORD"] = password
+	}
+}
+
+// WithUsername sets the initial username to be created when the container starts
+// It is used in conjunction with WithPassword to set a user and its password.
+// It will create the specified user with superuser power.
+// If it is not specified, then the default user of clickhouse will be used.
+func WithUsername(user string) testcontainers.CustomizeRequestOption {
+	return func(req *testcontainers.GenericContainerRequest) {
+		if user == "" {
+			user = defaultUser
+		}
+
+		req.Env["CLICKHOUSE_USER"] = user
+	}
+}
+
+// RunContainer creates an instance of the ClickHouse container type
+func RunContainer(ctx context.Context, opts ...testcontainers.ContainerCustomizer) (*ClickHouseContainer, error) {
+	req := testcontainers.ContainerRequest{
+		Image: defaultImage,
+		Env: map[string]string{
+			"CLICKHOUSE_USER":     defaultUser,
+			"CLICKHOUSE_PASSWORD": defaultUser,
+			"CLICKHOUSE_DB":       defaultDatabaseName,
+		},
+		ExposedPorts: []string{httpPort.Port(), nativePort.Port()},
+		WaitingFor: wait.ForAll(
+			wait.NewHTTPStrategy("/").WithPort(httpPort).WithStatusCodeMatcher(func(status int) bool {
+				return status == 200
+			}),
+		),
+	}
+
+	genericContainerReq := testcontainers.GenericContainerRequest{
+		ContainerRequest: req,
+		Started:          true,
+	}
+
+	for _, opt := range opts {
+		opt.Customize(&genericContainerReq)
+	}
+
+	container, err := testcontainers.GenericContainer(ctx, genericContainerReq)
+	if err != nil {
+		return nil, err
+	}
+
+	user := req.Env["CLICKHOUSE_USER"]
+	password := req.Env["CLICKHOUSE_PASSWORD"]
+	dbName := req.Env["CLICKHOUSE_DB"]
+
+	return &ClickHouseContainer{Container: container, dbName: dbName, password: password, user: user}, nil
+}