Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Add Cassandra module #1726

Merged
merged 7 commits into from
Oct 11, 2023
Merged
Show file tree
Hide file tree
Changes from 4 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
7 changes: 7 additions & 0 deletions .github/dependabot.yml
Expand Up @@ -65,6 +65,13 @@ updates:
day: sunday
open-pull-requests-limit: 3
rebase-strategy: disabled
- package-ecosystem: gomod
directory: /modules/cassandra
schedule:
interval: monthly
day: sunday
open-pull-requests-limit: 3
rebase-strategy: disabled
- package-ecosystem: gomod
directory: /modules/clickhouse
schedule:
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/ci.yml
Expand Up @@ -106,7 +106,7 @@ jobs:
matrix:
go-version: [1.20.x, 1.x]
platform: [ubuntu-latest, macos-latest]
module: [artemis, clickhouse, compose, couchbase, elasticsearch, gcloud, k3s, kafka, localstack, mariadb, mongodb, mysql, nats, neo4j, postgres, pulsar, rabbitmq, redis, redpanda, vault]
module: [artemis, cassandra, clickhouse, compose, couchbase, elasticsearch, gcloud, k3s, kafka, localstack, mariadb, mongodb, mysql, nats, neo4j, postgres, pulsar, rabbitmq, redis, redpanda, vault]
uses: ./.github/workflows/ci-test-go.yml
with:
go-version: ${{ matrix.go-version }}
Expand Down
4 changes: 4 additions & 0 deletions .vscode/.testcontainers-go.code-workspace
Expand Up @@ -25,6 +25,10 @@
"name": "module / artemis",
"path": "../modules/artemis"
},
{
"name": "module / cassandra",
"path": "../modules/cassandra"
},
{
"name": "module / clickhouse",
"path": "../modules/clickhouse"
Expand Down
73 changes: 73 additions & 0 deletions docs/modules/cassandra.md
@@ -0,0 +1,73 @@
# Cassandra

Not available until the next release of testcontainers-go <a href="https://github.com/testcontainers/testcontainers-go"><span class="tc-version">:material-tag: main</span></a>

## Introduction

The Testcontainers module for Cassandra.

## Adding this module to your project dependencies

Please run the following command to add the Cassandra module to your Go dependencies:

```
go get github.com/testcontainers/testcontainers-go/modules/cassandra
```

## Usage example

<!--codeinclude-->
[Creating a Cassandra container](../../modules/cassandra/examples_test.go) inside_block:runCassandraContainer
<!--/codeinclude-->

## Module reference

The Cassandra module exposes one entrypoint function to create the Cassandra container, and this function receives two parameters:

```golang
func RunContainer(ctx context.Context, opts ...testcontainers.ContainerCustomizer) (*CassandraContainer, error)
```

- `context.Context`, the Go context.
- `testcontainers.ContainerCustomizer`, a variadic argument for passing options.

### Container Options

When starting the Cassandra container, you can pass options in a variadic way to configure it.

#### Image

If you need to set a different Cassandra Docker image, you can use `testcontainers.WithImage` with a valid Docker image
for Cassandra. E.g. `testcontainers.WithImage("cassandra:4.1.3")`.

{% include "../features/common_functional_options.md" %}

#### Init Scripts

If you would like to do additional initialization in the Cassandra container, add one or more `*.cql` or `*.sh` scripts to the container request with the `WithInitScripts` function.
Those files will be copied after the container is created but before it's started under root directory.

An example of a `*.sh` script that creates a keyspace and table is shown below:

<!--codeinclude-->
[Init script content](../../modules/cassandra/testdata/init.sh)
<!--/codeinclude-->

#### Database configuration

In the case you have a custom config file for Cassandra, it's possible to copy that file into the container before it's started, using the `WithConfigFile(cfgPath string)` function.

!!!warning
You should provide a valid Cassandra configuration file, otherwise the container will fail to start.

### Container Methods

The Cassandra container exposes the following methods:

#### ConnectionHost

This method returns the host and port of the Cassandra container, using the default, `9042/tcp` port. E.g. `localhost:9042`

<!--codeinclude-->
[Get connection host](../../modules/cassandra/cassandra_test.go) inside_block:connectionHost
<!--/codeinclude-->
1 change: 1 addition & 0 deletions mkdocs.yml
Expand Up @@ -64,6 +64,7 @@ nav:
- Modules:
- modules/index.md
- modules/artemis.md
- modules/cassandra.md
- modules/clickhouse.md
- modules/couchbase.md
- modules/elasticsearch.md
Expand Down
5 changes: 5 additions & 0 deletions modules/cassandra/Makefile
@@ -0,0 +1,5 @@
include ../../commons-test.mk

.PHONY: test
test:
$(MAKE) test-cassandra
117 changes: 117 additions & 0 deletions modules/cassandra/cassandra.go
@@ -0,0 +1,117 @@
package cassandra

import (
"context"
"io"
"path/filepath"
"strings"

"github.com/docker/go-connections/nat"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Gci shall require a new line here.
Which IDE are you using ?

Copy link
Contributor Author

@anilsenay anilsenay Oct 9, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Im using vscode but also i checked in GoLand, it did not do anything or warn about it. But I added new line manually. Thank you

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mdelapenya , shall an issue be created ? This is the second module where the linting is not applied.
@anilsenay , Is golangci-lint installed on your machine ?
In vscode, are you using the workspace configuration from here ?
In your settings, do you have this:

"go.lintTool": "golangci-lint"

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if we could run golangci-lint when a module is generated, but instead of using the binary (see Makefile) calling the right Docker image to avoid having to install it on every developer's machine.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not every machine has docker, on windows for exemple, it's not installed by default.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, but installing docker is easier than installing a binary for linting. In any case, I'm biased from my past experience in a release engineering team, where we dockerised everything to run smoothly on the CI 😅


"github.com/testcontainers/testcontainers-go"
"github.com/testcontainers/testcontainers-go/wait"
)

const (
port = nat.Port("9042/tcp")
)

// CassandraContainer represents the Cassandra container type used in the module
type CassandraContainer struct {
testcontainers.Container
}

// ConnectionHost returns the host and port of the clickhouse container, using the default, native 9000 port, and
// obtaining the host and exposed port from the container
func (c *CassandraContainer) ConnectionHost(ctx context.Context) (string, error) {
host, err := c.Host(ctx)
if err != nil {
return "", err
}

port, err := c.MappedPort(ctx, port)
if err != nil {
return "", err
}

return host + ":" + port.Port(), nil
}

// WithConfigFile sets the YAML config file to be used for the cassandra container
// It will also set the "configFile" parameter to the path of the config file
// as a command line argument to the container.
func WithConfigFile(configFile string) testcontainers.CustomizeRequestOption {
return func(req *testcontainers.GenericContainerRequest) {
cf := testcontainers.ContainerFile{
HostFilePath: configFile,
ContainerFilePath: "/etc/cassandra/cassandra.yaml",
FileMode: 0o755,
}
req.Files = append(req.Files, cf)
}
}

// WithInitScripts sets the init cassandra queries to be run when the container starts
func WithInitScripts(scripts ...string) testcontainers.CustomizeRequestOption {
return func(req *testcontainers.GenericContainerRequest) {
var initScripts []testcontainers.ContainerFile
for _, script := range scripts {
cf := testcontainers.ContainerFile{
HostFilePath: script,
ContainerFilePath: "/" + filepath.Base(script),
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cassandra docker image does not support /docker-entrypoint-initdb.d/ so I put script files in root directory and in line 105 they will be executed via cqlsh or sh command by file extension. Is this a valid solution? What do you think?

FileMode: 0o755,
}
initScripts = append(initScripts, cf)
}
req.Files = append(req.Files, initScripts...)
mdelapenya marked this conversation as resolved.
Show resolved Hide resolved
}
}

// RunContainer creates an instance of the Cassandra container type
func RunContainer(ctx context.Context, opts ...testcontainers.ContainerCustomizer) (*CassandraContainer, error) {
req := testcontainers.ContainerRequest{
Image: "cassandra:4.1.3",
ExposedPorts: []string{string(port)},
Env: map[string]string{
"CASSANDRA_SNITCH": "GossipingPropertyFileSnitch",
"JVM_OPTS": "-Dcassandra.skip_wait_for_gossip_to_settle=0 -Dcassandra.initial_token=0",
"HEAP_NEWSIZE": "128M",
"MAX_HEAP_SIZE": "1024M",
mdelapenya marked this conversation as resolved.
Show resolved Hide resolved
"CASSANDRA_ENDPOINT_SNITCH": "GossipingPropertyFileSnitch",
"CASSANDRA_DC": "datacenter1",
},
WaitingFor: wait.ForAll(
wait.ForListeningPort(port),
wait.ForExec([]string{"cqlsh", "-e", "SELECT release_version FROM system.local"}).WithResponseMatcher(func(body io.Reader) bool {
data, _ := io.ReadAll(body)
return strings.Contains(string(data), "4.1.3")
anilsenay marked this conversation as resolved.
Show resolved Hide resolved
}),
),
}

genericContainerReq := testcontainers.GenericContainerRequest{
ContainerRequest: req,
Started: true,
}

for _, opt := range opts {
opt.Customize(&genericContainerReq)
}

container, err := testcontainers.GenericContainer(ctx, genericContainerReq)
if err != nil {
return nil, err
}

if len(genericContainerReq.Files) > 0 {
for _, file := range genericContainerReq.Files {
if strings.HasSuffix(file.ContainerFilePath, ".cql") {
container.Exec(ctx, []string{"cqlsh", "-f", file.ContainerFilePath})
} else if strings.HasSuffix(file.ContainerFilePath, ".sh") {
container.Exec(ctx, []string{"/bin/sh", file.ContainerFilePath})
}
}
}
anilsenay marked this conversation as resolved.
Show resolved Hide resolved

return &CassandraContainer{Container: container}, nil
}
147 changes: 147 additions & 0 deletions modules/cassandra/cassandra_test.go
@@ -0,0 +1,147 @@
package cassandra

import (
"context"
"testing"

"github.com/gocql/gocql"
"github.com/stretchr/testify/assert"
)

type Test struct {
Id uint64
Name string
}

func TestCassandra(t *testing.T) {
ctx := context.Background()

container, err := RunContainer(ctx)
if err != nil {
t.Fatal(err)
}

// Clean up the container after the test is complete
t.Cleanup(func() {
assert.NoError(t, container.Terminate(ctx))
})

// connectionString {
connectionHost, err := container.ConnectionHost(ctx)
// }
assert.NoError(t, err)

cluster := gocql.NewCluster(connectionHost)
session, err := cluster.CreateSession()
if err != nil {
t.Fatal(err)
}
defer session.Close()

// perform assertions
err = session.Query("CREATE KEYSPACE test_keyspace WITH REPLICATION = {'class' : 'SimpleStrategy', 'replication_factor' : 1}").Exec()
assert.NoError(t, err)
err = session.Query("CREATE TABLE test_keyspace.test_table (id int PRIMARY KEY, name text)").Exec()
assert.NoError(t, err)

err = session.Query("INSERT INTO test_keyspace.test_table (id, name) VALUES (1, 'NAME')").Exec()
assert.NoError(t, err)

var test Test
err = session.Query("SELECT id, name FROM test_keyspace.test_table WHERE id=1").Scan(&test.Id, &test.Name)
assert.NoError(t, err)
assert.Equal(t, Test{Id: 1, Name: "NAME"}, test)
}

func TestCassandraWithConfigFile(t *testing.T) {
ctx := context.Background()

container, err := RunContainer(ctx, WithConfigFile("testdata/config.yaml"))
anilsenay marked this conversation as resolved.
Show resolved Hide resolved
if err != nil {
t.Fatal(err)
}

// Clean up the container after the test is complete
t.Cleanup(func() {
assert.NoError(t, container.Terminate(ctx))
})

connectionHost, err := container.ConnectionHost(ctx)
assert.NoError(t, err)

cluster := gocql.NewCluster(connectionHost)
session, err := cluster.CreateSession()
if err != nil {
t.Fatal(err)
}
defer session.Close()

var result string
err = session.Query("SELECT cluster_name FROM system.local").Scan(&result)
assert.NoError(t, err)
assert.Equal(t, "My Cluster", result)
}

func TestCassandraWithInitScripts(t *testing.T) {
t.Run("with init cql script", func(t *testing.T) {
ctx := context.Background()

// withInitScripts {
container, err := RunContainer(ctx, WithInitScripts("testdata/init.cql"))
anilsenay marked this conversation as resolved.
Show resolved Hide resolved
// }
if err != nil {
t.Fatal(err)
}

// Clean up the container after the test is complete
t.Cleanup(func() {
assert.NoError(t, container.Terminate(ctx))
})

// connectionHost {
connectionHost, err := container.ConnectionHost(ctx)
// }
assert.NoError(t, err)

cluster := gocql.NewCluster(connectionHost)
session, err := cluster.CreateSession()
if err != nil {
t.Fatal(err)
}
defer session.Close()

var test Test
err = session.Query("SELECT id, name FROM test_keyspace.test_table WHERE id=1").Scan(&test.Id, &test.Name)
assert.NoError(t, err)
assert.Equal(t, Test{Id: 1, Name: "NAME"}, test)
})

t.Run("with init bash script", func(t *testing.T) {
ctx := context.Background()

container, err := RunContainer(ctx, WithInitScripts("testdata/init.sh"))
if err != nil {
t.Fatal(err)
}

// Clean up the container after the test is complete
t.Cleanup(func() {
assert.NoError(t, container.Terminate(ctx))
})

connectionHost, err := container.ConnectionHost(ctx)
assert.NoError(t, err)

cluster := gocql.NewCluster(connectionHost)
session, err := cluster.CreateSession()
if err != nil {
t.Fatal(err)
}
defer session.Close()

var test Test
err = session.Query("SELECT id, name FROM init_sh_keyspace.test_table WHERE id=1").Scan(&test.Id, &test.Name)
assert.NoError(t, err)
assert.Equal(t, Test{Id: 1, Name: "NAME"}, test)
})
}