Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added Kafka (KRaft Mode) module #1034

Closed
wants to merge 1 commit into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
10 changes: 8 additions & 2 deletions .github/dependabot.yml
Original file line number Diff line number Diff line change
Expand Up @@ -84,6 +84,12 @@ updates:
interval: monthly
open-pull-requests-limit: 3
rebase-strategy: disabled
- package-ecosystem: gomod
directory: /modules/kafka
schedule:
interval: monthly
open-pull-requests-limit: 3
rebase-strategy: disabled
- package-ecosystem: gomod
directory: /modules/localstack
schedule:
Expand All @@ -97,13 +103,13 @@ updates:
open-pull-requests-limit: 3
rebase-strategy: disabled
- package-ecosystem: gomod
directory: /modules/postgres
directory: /modules/neo4j
schedule:
interval: monthly
open-pull-requests-limit: 3
rebase-strategy: disabled
- package-ecosystem: gomod
directory: /modules/neo4j
directory: /modules/postgres
schedule:
interval: monthly
open-pull-requests-limit: 3
Expand Down
56 changes: 56 additions & 0 deletions .github/workflows/module-kafka.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
name: Kafka module pipeline

on:
push:
paths-ignore:
- 'mkdocs.yml'
- 'docs/**'
- 'README.md'
pull_request:
paths-ignore:
- 'mkdocs.yml'
- 'docs/**'
- 'README.md'

concurrency:
group: ${{ github.workflow }}-${{ github.head_ref || github.sha }}
cancel-in-progress: true

jobs:
test-kafka:
strategy:
matrix:
go-version: [1.19.x, 1.x]
runs-on: "ubuntu-latest"
steps:

- name: Set up Go
uses: actions/setup-go@v3
with:
go-version: ${{ matrix.go-version }}
id: go

- name: Check out code into the Go module directory
uses: actions/checkout@v3

- name: modVerify
working-directory: ./modules/kafka
run: go mod verify

- name: modTidy
working-directory: ./modules/kafka
run: make tools-tidy

- name: gotestsum
working-directory: ./modules/kafka
run: make test-unit

- name: Run checker
run: |
./scripts/check_environment.sh

- name: Test Summary
uses: test-summary/action@4ee9ece4bca777a38f05c8fc578ac2007fe266f7
with:
paths: "**/TEST-kafka*.xml"
if: always()
35 changes: 35 additions & 0 deletions docs/modules/kafka.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
# Kafka

Testcontainers can automatically start an Apache Kafka container in KRaft (Kafka Raft)
mode. The container uses the official Confluent Kafka Platform image.

## Adding this module to your project dependencies

Please run the following command to add the Kafka module to your Go dependencies:

```
go get github.com/testcontainers/testcontainers-go/modules/kafka
```

## Usage example

Create a `Kafka` container to use it in your tests:

<!--codeinclude-->
[Creating a Kafka container](../../modules/kafka/kafka.go)
<!--/codeinclude-->

After startup of the container, the broker address can be retrieved as following:

```go
brokers, err := container.Brokers(context.Background())
if err != nil {
// ... error handling
}
```

A full example with Sarama producer and consumer:

<!--codeinclude-->
[Test for a Kafka container](../../modules/kafka/kafka_test.go) inside_block:TestStartContainer
<!--/codeinclude-->
3 changes: 2 additions & 1 deletion mkdocs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -54,10 +54,11 @@ nav:
- Modules:
- modules/index.md
- modules/couchbase.md
- modules/kafka.md
- modules/localstack.md
- modules/mysql.md
- modules/postgres.md
- modules/neo4j.md
- modules/postgres.md
- modules/pulsar.md
- modules/redis.md
- modules/vault.md
Expand Down
5 changes: 5 additions & 0 deletions modules/kafka/Makefile
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
include ../../commons-test.mk

.PHONY: test
test:
$(MAKE) test-kafka
72 changes: 72 additions & 0 deletions modules/kafka/go.mod
Original file line number Diff line number Diff line change
@@ -0,0 +1,72 @@
module github.com/testcontainers/testcontainers-go/modules/kafka

go 1.19

require (
github.com/Shopify/sarama v1.38.1
github.com/docker/go-connections v0.4.0
github.com/stretchr/testify v1.8.2
github.com/testcontainers/testcontainers-go v0.19.0
gotest.tools/gotestsum v1.9.0
)

require (
github.com/Azure/go-ansiterm v0.0.0-20210617225240-d185dfc1b5a1 // indirect
github.com/Microsoft/go-winio v0.5.2 // indirect
github.com/cenkalti/backoff/v4 v4.2.0 // indirect
github.com/containerd/containerd v1.6.19 // indirect
github.com/cpuguy83/dockercfg v0.3.1 // indirect
github.com/davecgh/go-spew v1.1.1 // indirect
github.com/dnephin/pflag v1.0.7 // indirect
github.com/docker/distribution v2.8.1+incompatible // indirect
github.com/docker/docker v23.0.2+incompatible // indirect
github.com/docker/go-units v0.5.0 // indirect
github.com/eapache/go-resiliency v1.3.0 // indirect
github.com/eapache/go-xerial-snappy v0.0.0-20230111030713-bf00bc1b83b6 // indirect
github.com/eapache/queue v1.1.0 // indirect
github.com/fatih/color v1.13.0 // indirect
github.com/fsnotify/fsnotify v1.5.4 // indirect
github.com/gogo/protobuf v1.3.2 // indirect
github.com/golang/protobuf v1.5.2 // indirect
github.com/golang/snappy v0.0.4 // indirect
github.com/google/shlex v0.0.0-20191202100458-e7afc7fbc510 // indirect
github.com/google/uuid v1.3.0 // indirect
github.com/hashicorp/errwrap v1.1.0 // indirect
github.com/hashicorp/go-multierror v1.1.1 // indirect
github.com/hashicorp/go-uuid v1.0.3 // indirect
github.com/jcmturner/aescts/v2 v2.0.0 // indirect
github.com/jcmturner/dnsutils/v2 v2.0.0 // indirect
github.com/jcmturner/gofork v1.7.6 // indirect
github.com/jcmturner/gokrb5/v8 v8.4.3 // indirect
github.com/jcmturner/rpc/v2 v2.0.3 // indirect
github.com/klauspost/compress v1.15.14 // indirect
github.com/magiconair/properties v1.8.7 // indirect
github.com/mattn/go-colorable v0.1.12 // indirect
github.com/mattn/go-isatty v0.0.16 // indirect
github.com/moby/patternmatcher v0.5.0 // indirect
github.com/moby/sys/sequential v0.5.0 // indirect
github.com/moby/term v0.0.0-20221128092401-c43b287e0e0f // indirect
github.com/morikuni/aec v1.0.0 // indirect
github.com/opencontainers/go-digest v1.0.0 // indirect
github.com/opencontainers/image-spec v1.1.0-rc2 // indirect
github.com/opencontainers/runc v1.1.5 // indirect
github.com/pierrec/lz4/v4 v4.1.17 // indirect
github.com/pkg/errors v0.9.1 // indirect
github.com/pmezard/go-difflib v1.0.0 // indirect
github.com/rcrowley/go-metrics v0.0.0-20201227073835-cf1acfcdf475 // indirect
github.com/sirupsen/logrus v1.9.0 // indirect
golang.org/x/crypto v0.0.0-20220722155217-630584e8d5aa // indirect
golang.org/x/mod v0.6.0-dev.0.20220419223038-86c51ed26bb4 // indirect
golang.org/x/net v0.7.0 // indirect
golang.org/x/sync v0.1.0 // indirect
golang.org/x/sys v0.6.0 // indirect
golang.org/x/term v0.5.0 // indirect
golang.org/x/tools v0.1.12 // indirect
google.golang.org/genproto v0.0.0-20220617124728-180714bec0ad // indirect
google.golang.org/grpc v1.47.0 // indirect
google.golang.org/protobuf v1.28.0 // indirect
gopkg.in/yaml.v3 v3.0.1 // indirect
gotest.tools/v3 v3.4.0 // indirect
)

replace github.com/testcontainers/testcontainers-go => ../..