-
-
Notifications
You must be signed in to change notification settings - Fork 431
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat: Add Cassandra module #1726
Merged
Merged
Changes from 4 commits
Commits
Show all changes
7 commits
Select commit
Hold shift + click to select a range
9ba393e
Add cassandra module
anilsenay 0b80cdd
Merge branch 'main' into cassandra-module
anilsenay f086e9a
Add new line before testcontainers imports
anilsenay 3daf05b
Merge branch 'cassandra-module' of github.com:anilsenay/testcontainer…
anilsenay 173e102
Refactor the way for running init scripts
anilsenay e551301
Merge branch 'main' into cassandra-module
anilsenay 70992e2
fix: typo
mdelapenya File filter
Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,73 @@ | ||
# Cassandra | ||
|
||
Not available until the next release of testcontainers-go <a href="https://github.com/testcontainers/testcontainers-go"><span class="tc-version">:material-tag: main</span></a> | ||
|
||
## Introduction | ||
|
||
The Testcontainers module for Cassandra. | ||
|
||
## Adding this module to your project dependencies | ||
|
||
Please run the following command to add the Cassandra module to your Go dependencies: | ||
|
||
``` | ||
go get github.com/testcontainers/testcontainers-go/modules/cassandra | ||
``` | ||
|
||
## Usage example | ||
|
||
<!--codeinclude--> | ||
[Creating a Cassandra container](../../modules/cassandra/examples_test.go) inside_block:runCassandraContainer | ||
<!--/codeinclude--> | ||
|
||
## Module reference | ||
|
||
The Cassandra module exposes one entrypoint function to create the Cassandra container, and this function receives two parameters: | ||
|
||
```golang | ||
func RunContainer(ctx context.Context, opts ...testcontainers.ContainerCustomizer) (*CassandraContainer, error) | ||
``` | ||
|
||
- `context.Context`, the Go context. | ||
- `testcontainers.ContainerCustomizer`, a variadic argument for passing options. | ||
|
||
### Container Options | ||
|
||
When starting the Cassandra container, you can pass options in a variadic way to configure it. | ||
|
||
#### Image | ||
|
||
If you need to set a different Cassandra Docker image, you can use `testcontainers.WithImage` with a valid Docker image | ||
for Cassandra. E.g. `testcontainers.WithImage("cassandra:4.1.3")`. | ||
|
||
{% include "../features/common_functional_options.md" %} | ||
|
||
#### Init Scripts | ||
|
||
If you would like to do additional initialization in the Cassandra container, add one or more `*.cql` or `*.sh` scripts to the container request with the `WithInitScripts` function. | ||
Those files will be copied after the container is created but before it's started under root directory. | ||
|
||
An example of a `*.sh` script that creates a keyspace and table is shown below: | ||
|
||
<!--codeinclude--> | ||
[Init script content](../../modules/cassandra/testdata/init.sh) | ||
<!--/codeinclude--> | ||
|
||
#### Database configuration | ||
|
||
In the case you have a custom config file for Cassandra, it's possible to copy that file into the container before it's started, using the `WithConfigFile(cfgPath string)` function. | ||
|
||
!!!warning | ||
You should provide a valid Cassandra configuration file, otherwise the container will fail to start. | ||
|
||
### Container Methods | ||
|
||
The Cassandra container exposes the following methods: | ||
|
||
#### ConnectionHost | ||
|
||
This method returns the host and port of the Cassandra container, using the default, `9042/tcp` port. E.g. `localhost:9042` | ||
|
||
<!--codeinclude--> | ||
[Get connection host](../../modules/cassandra/cassandra_test.go) inside_block:connectionHost | ||
<!--/codeinclude--> |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,5 @@ | ||
include ../../commons-test.mk | ||
|
||
.PHONY: test | ||
test: | ||
$(MAKE) test-cassandra |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,117 @@ | ||
package cassandra | ||
|
||
import ( | ||
"context" | ||
"io" | ||
"path/filepath" | ||
"strings" | ||
|
||
"github.com/docker/go-connections/nat" | ||
|
||
"github.com/testcontainers/testcontainers-go" | ||
"github.com/testcontainers/testcontainers-go/wait" | ||
) | ||
|
||
const ( | ||
port = nat.Port("9042/tcp") | ||
) | ||
|
||
// CassandraContainer represents the Cassandra container type used in the module | ||
type CassandraContainer struct { | ||
testcontainers.Container | ||
} | ||
|
||
// ConnectionHost returns the host and port of the clickhouse container, using the default, native 9000 port, and | ||
// obtaining the host and exposed port from the container | ||
func (c *CassandraContainer) ConnectionHost(ctx context.Context) (string, error) { | ||
host, err := c.Host(ctx) | ||
if err != nil { | ||
return "", err | ||
} | ||
|
||
port, err := c.MappedPort(ctx, port) | ||
if err != nil { | ||
return "", err | ||
} | ||
|
||
return host + ":" + port.Port(), nil | ||
} | ||
|
||
// WithConfigFile sets the YAML config file to be used for the cassandra container | ||
// It will also set the "configFile" parameter to the path of the config file | ||
// as a command line argument to the container. | ||
func WithConfigFile(configFile string) testcontainers.CustomizeRequestOption { | ||
return func(req *testcontainers.GenericContainerRequest) { | ||
cf := testcontainers.ContainerFile{ | ||
HostFilePath: configFile, | ||
ContainerFilePath: "/etc/cassandra/cassandra.yaml", | ||
FileMode: 0o755, | ||
} | ||
req.Files = append(req.Files, cf) | ||
} | ||
} | ||
|
||
// WithInitScripts sets the init cassandra queries to be run when the container starts | ||
func WithInitScripts(scripts ...string) testcontainers.CustomizeRequestOption { | ||
return func(req *testcontainers.GenericContainerRequest) { | ||
var initScripts []testcontainers.ContainerFile | ||
for _, script := range scripts { | ||
cf := testcontainers.ContainerFile{ | ||
HostFilePath: script, | ||
ContainerFilePath: "/" + filepath.Base(script), | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. cassandra docker image does not support |
||
FileMode: 0o755, | ||
} | ||
initScripts = append(initScripts, cf) | ||
} | ||
req.Files = append(req.Files, initScripts...) | ||
mdelapenya marked this conversation as resolved.
Show resolved
Hide resolved
|
||
} | ||
} | ||
|
||
// RunContainer creates an instance of the Cassandra container type | ||
func RunContainer(ctx context.Context, opts ...testcontainers.ContainerCustomizer) (*CassandraContainer, error) { | ||
req := testcontainers.ContainerRequest{ | ||
Image: "cassandra:4.1.3", | ||
ExposedPorts: []string{string(port)}, | ||
Env: map[string]string{ | ||
"CASSANDRA_SNITCH": "GossipingPropertyFileSnitch", | ||
"JVM_OPTS": "-Dcassandra.skip_wait_for_gossip_to_settle=0 -Dcassandra.initial_token=0", | ||
"HEAP_NEWSIZE": "128M", | ||
"MAX_HEAP_SIZE": "1024M", | ||
mdelapenya marked this conversation as resolved.
Show resolved
Hide resolved
|
||
"CASSANDRA_ENDPOINT_SNITCH": "GossipingPropertyFileSnitch", | ||
"CASSANDRA_DC": "datacenter1", | ||
}, | ||
WaitingFor: wait.ForAll( | ||
wait.ForListeningPort(port), | ||
wait.ForExec([]string{"cqlsh", "-e", "SELECT release_version FROM system.local"}).WithResponseMatcher(func(body io.Reader) bool { | ||
data, _ := io.ReadAll(body) | ||
return strings.Contains(string(data), "4.1.3") | ||
anilsenay marked this conversation as resolved.
Show resolved
Hide resolved
|
||
}), | ||
), | ||
} | ||
|
||
genericContainerReq := testcontainers.GenericContainerRequest{ | ||
ContainerRequest: req, | ||
Started: true, | ||
} | ||
|
||
for _, opt := range opts { | ||
opt.Customize(&genericContainerReq) | ||
} | ||
|
||
container, err := testcontainers.GenericContainer(ctx, genericContainerReq) | ||
if err != nil { | ||
return nil, err | ||
} | ||
|
||
if len(genericContainerReq.Files) > 0 { | ||
for _, file := range genericContainerReq.Files { | ||
if strings.HasSuffix(file.ContainerFilePath, ".cql") { | ||
container.Exec(ctx, []string{"cqlsh", "-f", file.ContainerFilePath}) | ||
} else if strings.HasSuffix(file.ContainerFilePath, ".sh") { | ||
container.Exec(ctx, []string{"/bin/sh", file.ContainerFilePath}) | ||
} | ||
} | ||
} | ||
anilsenay marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
return &CassandraContainer{Container: container}, nil | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,147 @@ | ||
package cassandra | ||
|
||
import ( | ||
"context" | ||
"testing" | ||
|
||
"github.com/gocql/gocql" | ||
"github.com/stretchr/testify/assert" | ||
) | ||
|
||
type Test struct { | ||
Id uint64 | ||
Name string | ||
} | ||
|
||
func TestCassandra(t *testing.T) { | ||
ctx := context.Background() | ||
|
||
container, err := RunContainer(ctx) | ||
if err != nil { | ||
t.Fatal(err) | ||
} | ||
|
||
// Clean up the container after the test is complete | ||
t.Cleanup(func() { | ||
assert.NoError(t, container.Terminate(ctx)) | ||
}) | ||
|
||
// connectionString { | ||
connectionHost, err := container.ConnectionHost(ctx) | ||
// } | ||
assert.NoError(t, err) | ||
|
||
cluster := gocql.NewCluster(connectionHost) | ||
session, err := cluster.CreateSession() | ||
if err != nil { | ||
t.Fatal(err) | ||
} | ||
defer session.Close() | ||
|
||
// perform assertions | ||
err = session.Query("CREATE KEYSPACE test_keyspace WITH REPLICATION = {'class' : 'SimpleStrategy', 'replication_factor' : 1}").Exec() | ||
assert.NoError(t, err) | ||
err = session.Query("CREATE TABLE test_keyspace.test_table (id int PRIMARY KEY, name text)").Exec() | ||
assert.NoError(t, err) | ||
|
||
err = session.Query("INSERT INTO test_keyspace.test_table (id, name) VALUES (1, 'NAME')").Exec() | ||
assert.NoError(t, err) | ||
|
||
var test Test | ||
err = session.Query("SELECT id, name FROM test_keyspace.test_table WHERE id=1").Scan(&test.Id, &test.Name) | ||
assert.NoError(t, err) | ||
assert.Equal(t, Test{Id: 1, Name: "NAME"}, test) | ||
} | ||
|
||
func TestCassandraWithConfigFile(t *testing.T) { | ||
ctx := context.Background() | ||
|
||
container, err := RunContainer(ctx, WithConfigFile("testdata/config.yaml")) | ||
anilsenay marked this conversation as resolved.
Show resolved
Hide resolved
|
||
if err != nil { | ||
t.Fatal(err) | ||
} | ||
|
||
// Clean up the container after the test is complete | ||
t.Cleanup(func() { | ||
assert.NoError(t, container.Terminate(ctx)) | ||
}) | ||
|
||
connectionHost, err := container.ConnectionHost(ctx) | ||
assert.NoError(t, err) | ||
|
||
cluster := gocql.NewCluster(connectionHost) | ||
session, err := cluster.CreateSession() | ||
if err != nil { | ||
t.Fatal(err) | ||
} | ||
defer session.Close() | ||
|
||
var result string | ||
err = session.Query("SELECT cluster_name FROM system.local").Scan(&result) | ||
assert.NoError(t, err) | ||
assert.Equal(t, "My Cluster", result) | ||
} | ||
|
||
func TestCassandraWithInitScripts(t *testing.T) { | ||
t.Run("with init cql script", func(t *testing.T) { | ||
ctx := context.Background() | ||
|
||
// withInitScripts { | ||
container, err := RunContainer(ctx, WithInitScripts("testdata/init.cql")) | ||
anilsenay marked this conversation as resolved.
Show resolved
Hide resolved
|
||
// } | ||
if err != nil { | ||
t.Fatal(err) | ||
} | ||
|
||
// Clean up the container after the test is complete | ||
t.Cleanup(func() { | ||
assert.NoError(t, container.Terminate(ctx)) | ||
}) | ||
|
||
// connectionHost { | ||
connectionHost, err := container.ConnectionHost(ctx) | ||
// } | ||
assert.NoError(t, err) | ||
|
||
cluster := gocql.NewCluster(connectionHost) | ||
session, err := cluster.CreateSession() | ||
if err != nil { | ||
t.Fatal(err) | ||
} | ||
defer session.Close() | ||
|
||
var test Test | ||
err = session.Query("SELECT id, name FROM test_keyspace.test_table WHERE id=1").Scan(&test.Id, &test.Name) | ||
assert.NoError(t, err) | ||
assert.Equal(t, Test{Id: 1, Name: "NAME"}, test) | ||
}) | ||
|
||
t.Run("with init bash script", func(t *testing.T) { | ||
ctx := context.Background() | ||
|
||
container, err := RunContainer(ctx, WithInitScripts("testdata/init.sh")) | ||
if err != nil { | ||
t.Fatal(err) | ||
} | ||
|
||
// Clean up the container after the test is complete | ||
t.Cleanup(func() { | ||
assert.NoError(t, container.Terminate(ctx)) | ||
}) | ||
|
||
connectionHost, err := container.ConnectionHost(ctx) | ||
assert.NoError(t, err) | ||
|
||
cluster := gocql.NewCluster(connectionHost) | ||
session, err := cluster.CreateSession() | ||
if err != nil { | ||
t.Fatal(err) | ||
} | ||
defer session.Close() | ||
|
||
var test Test | ||
err = session.Query("SELECT id, name FROM init_sh_keyspace.test_table WHERE id=1").Scan(&test.Id, &test.Name) | ||
assert.NoError(t, err) | ||
assert.Equal(t, Test{Id: 1, Name: "NAME"}, test) | ||
}) | ||
} |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Gci shall require a new line here.
Which IDE are you using ?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Im using vscode but also i checked in GoLand, it did not do anything or warn about it. But I added new line manually. Thank you
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@mdelapenya , shall an issue be created ? This is the second module where the linting is not applied.
@anilsenay , Is golangci-lint installed on your machine ?
In vscode, are you using the workspace configuration from here ?
In your settings, do you have this:
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I wonder if we could run golangci-lint when a module is generated, but instead of using the binary (see Makefile) calling the right Docker image to avoid having to install it on every developer's machine.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Not every machine has docker, on windows for exemple, it's not installed by default.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah, but installing docker is easier than installing a binary for linting. In any case, I'm biased from my past experience in a release engineering team, where we dockerised everything to run smoothly on the CI 😅