Add YugabyteDB module #4372

srinivasa-vasu · 2021-08-15T10:10:26Z

Every item on this list will require judgement by the Testcontainers core maintainers. Exceptions will sometimes be possible; items with should are more likely to be negotiable than those items with must.

Default docker image

Should be a Docker Hub official image, or published by a reputable source (ideally the company or organisation that officially supports the technology)
Should have a verifiable open source Dockerfile and a way to view the history of changes
MUST show general good practices regarding container image tagging - e.g. we do not use latest tags, and we do not use tags that may be mutated in the future
MUST be legal for Testcontainers developers and Testcontainers users to pull and use. Mechanisms exist to allow EULA acceptance to be signalled, but images that can be used without a licence are greatly preferred.

Module dependencies

The module should use as few dependencies as possible,
Regarding libraries, either:
they should be compileOnly if they are likely to already be on the classpath for users' tests (e.g. client libraries or drivers)
they can be implementation (and thus transitive dependencies) if they are very unlikely to conflict with users' dependencies.
If client libraries are used to test or use the module, these MUST be legal for Testcontainers developers and Testcontainers users to download and use.

API (e.g. MyModuleContainer class)

Favour doing the right thing, and least surprising thing, by default
Ensure that method and parameter names are easy to understand. Many users will ignore documentation, so IDE-based substitutes (autocompletion and Javadocs) should be intuitive.
The module's public API should only handle standard JDK data types and MUST not expose data types that come from compileOnly dependencies. This is to reduce the risk of compatibility problems with future versions of third party libraries.

Documentation
Every module MUST have a dedicated documentation page containing:

A high level overview
A usage example
If appropriate, basic API documentation or further usage guidelines
Dependency information
Acknowledgements, if appropriate
Consider that many users will not read the documentation pages - even if the first person to add it to a project does, people reading/updating the code in the future may not. Try and avoid the need for critical knowledge that is only present in documentation.

nchandrappa · 2021-12-03T21:14:59Z

@bsideup @kiview @rnorth Hi All, There is a lot of interest in our community to get YugabyteDB support in TestContainers project, please let us know what are steps required for getting the following PR merged and bringing support for YugabyteDB in Testcontianers project. Thanks, Nikhil

paskos · 2021-12-14T19:45:05Z

Hi,
We're about to start using yugabyte in our project. Is there an ETA on this PR ?

srinivasa-vasu · 2022-04-27T03:11:51Z

@bsideup @kiview @rnorth - Pls let us know if you need anything from our side to get this PR reviewed and merged. There is a lot of interest in the YB community to use this. Thnx.

eddumelendez

Thanks for the PR @srinivasa-vasu ! LGTM overall but WDYT about having YugabyteContainer only and provide something like withAPI(YugabyteAPI.YSQL), withAPI(YugabyteAPI.YCQL)?

docs/modules/databases/yugabytedb.md

modules/yugabytedb/src/main/java/org/testcontainers/containers/YugabyteContainerConstants.java

srinivasa-vasu · 2022-07-26T16:45:50Z

Thanks for the PR @srinivasa-vasu ! LGTM overall but WDYT about having YugabyteContainer only and provide something like withAPI(YugabyteAPI.YSQL), withAPI(YugabyteAPI.YCQL)?

@eddumelendez - Thanks for reviewing this. The APIs' are different, one is fully relational, and the other is semi and Cassandra compatible. Will end up with a lot of conditional blocks. Having it separate may be a cleaner abstraction.

eddumelendez · 2022-07-26T16:47:19Z

thanks for addressing the comments so fast! We will take a look at the changes

modules/yugabytedb/build.gradle

eddumelendez

leaving additional comments and still trying to figure it out how to handle one container class for both APIs.

...les/yugabytedb/src/test/java/org/testcontainers/junit/yugabytedb/YugabyteDBYSQLUnitTest.java

...les/yugabytedb/src/test/java/org/testcontainers/junit/yugabytedb/YugabyteDBYCQLUnitTest.java

modules/yugabytedb/src/main/java/org/testcontainers/containers/YugabyteDBYCQLContainer.java

eddumelendez · 2022-08-01T22:30:29Z

modules/yugabytedb/src/main/java/org/testcontainers/containers/YugabyteDBYCQLContainer.java

+	@Override
+	protected void containerIsStarted(InspectContainerResponse containerInfo) {
+		if (initScript != null) {
+			ScriptUtils.runInitScript(new YugabyteDBYCQLDelegate(getSessionBuilder()), initScript);
+		}
+	}


initScript is a very convenient method but there are existing tools such as liquibase-cassandra with this purpose and execInContainer() would help to perform similar operations. Doing so, we can remove YugabyteDBYCQLDelegate. WDYT @kiview ?

I've decoupled the Cassandra client library dependency but kept the API as is. Removed Cassandra dependency from YugabyteDBYCQLDelegate. This API may be useful for standalone services. Let me know WDYT?

modules/yugabytedb/src/main/java/org/testcontainers/containers/YugabyteDBYCQLContainer.java

modules/yugabytedb/src/main/java/org/testcontainers/containers/YugabyteDBYSQLContainer.java

eddumelendez · 2022-08-04T20:12:19Z

@srinivasa-vasu can you please add yugabyte to

.github/ISSUE_TEMPLATE/bug_report.yaml
.github/ISSUE_TEMPLATE/enhancement.yaml
.github/ISSUE_TEMPLATE/feature.yaml
.github/dependabot.yml
.github/labeler.yml

srinivasa-vasu · 2022-08-31T01:30:58Z

@eddumelendez - anything pending from my side?

eddumelendez

@srinivasa-vasu sorry for the delay. I have left a couple of comments about using assertj instead.

modules/yugabytedb/src/test/java/org/testcontainers/junit/yugabytedb/YugabyteDBYCQLTest.java

modules/yugabytedb/src/test/java/org/testcontainers/junit/yugabytedb/YugabyteDBYSQLTest.java

eddumelendez · 2022-10-17T19:23:20Z

modules/yugabytedb/src/main/java/org/testcontainers/containers/YugabyteDBYCQLContainer.java

+	/**
+	 * @param initScript path of the initialization script file
+	 * @return {@link YugabyteDBYCQLContainer} instance
+	 */
+	public YugabyteDBYCQLContainer withInitScript(String initScript) {
+		this.initScript = initScript;
+		return this;
+	}


I think this should not be supported because Testcontainers responsibility is about taking care of the service initialization. There are other tools which can help about keyspace and data management such as liquibase-cassandra. Also, that way we are avoiding being tight to a Cassandra library version. @kiview WDYT?

I agree with @eddumelendez. Adding such an API increases complexity more than necessary.

I've decoupled the Cassandra client library dependency but kept the API as is. This API may be useful for standalone services. Let me know WDYT?

eddumelendez

Just some suggestions in order to pass checks. Make sure to perform ./gradlew checkstyleMain checkstyleTest spotlessApply before pushing changes to make sure all checks are ok.

...ugabytedb/src/test/java/org/testcontainers/jdbc/yugabytedb/YugabyteDBYSQLJDBCDriverTest.java

eddumelendez · 2022-10-24T15:58:56Z

...abytedb/src/main/java/org/testcontainers/containers/strategy/YugabyteDBYCQLWaitStrategy.java

+ */
+@RequiredArgsConstructor
+@Slf4j
+public final class YugabyteDBYCQLWaitStrategy extends AbstractWaitStrategy {


this indeed now looks very convenient. I wonder if can do something similar to our existing CassandraContainer and rid of the cassandra driver dependency at all, at least for now.

@srinivasa-vasu to give you a little more context, on sql there are some issues due to how each database handles the scripts. We recommend using tools such as liquibase or flyway in order to execute scripts that are specialized for that kind of jobs. TBH, not sure if we will have similar issues with CQL and for that reason we will try to avoid it. IMO, we should avoid it and promote best practices around database management in general. No need to take action yet on this. @kiview WDYT?

Signed-off-by: Srinivasa Vasu <srinivasan.surprise@gmail.com>

…acking Signed-off-by: Srinivasa Vasu <srinivasan.surprise@gmail.com>

Signed-off-by: Srinivasa Vasu <srinivasan.surprise@gmail.com>

eddumelendez · 2022-11-15T18:28:39Z

thank you so much for your contribution @srinivasa-vasu ! This is now in main branch and it will be part of the next release.

srinivasa-vasu · 2022-11-16T13:00:07Z

Thanks @eddumelendez

srinivasa-vasu requested review from bsideup, kiview and rnorth as code owners August 15, 2021 10:10

srinivasa-vasu mentioned this pull request Aug 17, 2021

New module: YugabyteDB #4376

Closed

srinivasa-vasu force-pushed the yugabyte branch from a4b6e20 to 45273e7 Compare November 14, 2021 03:06

srinivasa-vasu force-pushed the yugabyte branch from b0c70e2 to e1dc326 Compare March 9, 2022 17:10

eddumelendez requested changes Jul 25, 2022

View reviewed changes

docs/modules/databases/yugabytedb.md Outdated Show resolved Hide resolved

docs/modules/databases/yugabytedb.md Outdated Show resolved Hide resolved

modules/yugabytedb/src/main/java/org/testcontainers/containers/YugabyteContainerConstants.java Outdated Show resolved Hide resolved

srinivasa-vasu force-pushed the yugabyte branch from 25631e8 to 3336c63 Compare July 26, 2022 16:39

srinivasa-vasu requested a review from a team as a code owner July 26, 2022 16:39

github-actions bot added the type/docs label Jul 26, 2022

eddumelendez assigned eddumelendez and kiview Jul 26, 2022

eddumelendez added the type/new module label Jul 26, 2022

eddumelendez reviewed Jul 28, 2022

View reviewed changes

modules/yugabytedb/build.gradle Outdated Show resolved Hide resolved

srinivasa-vasu force-pushed the yugabyte branch from e795d36 to ab4aa7c Compare July 28, 2022 18:02

eddumelendez requested changes Aug 1, 2022

View reviewed changes

srinivasa-vasu force-pushed the yugabyte branch from ab4aa7c to 47a6e03 Compare August 3, 2022 17:39

eddumelendez reviewed Aug 4, 2022

View reviewed changes

modules/yugabytedb/src/main/java/org/testcontainers/containers/YugabyteDBYCQLContainer.java Outdated Show resolved Hide resolved

modules/yugabytedb/src/main/java/org/testcontainers/containers/YugabyteDBYSQLContainer.java Outdated Show resolved Hide resolved

srinivasa-vasu force-pushed the yugabyte branch from a91af2d to bbaa4aa Compare August 5, 2022 01:28

eddumelendez changed the title ~~yugabytedb :: Testcontainer implementation for YSQL and YCQL APIs :: initial-commit~~ Add YugabyteDB module Aug 6, 2022

srinivasa-vasu force-pushed the yugabyte branch from 435c987 to c7d68ea Compare September 18, 2022 16:16

eddumelendez requested changes Oct 17, 2022

View reviewed changes

srinivasa-vasu force-pushed the yugabyte branch from c7d68ea to dba2a50 Compare October 23, 2022 07:02

eddumelendez reviewed Oct 24, 2022

View reviewed changes

srinivasa-vasu force-pushed the yugabyte branch from dba2a50 to 0af5d61 Compare October 25, 2022 02:20

eddumelendez previously approved these changes Nov 8, 2022

View reviewed changes

srinivasa-vasu added 17 commits November 11, 2022 12:10

yugabytedb - initial commit

20889fd

Signed-off-by: Srinivasa Vasu <srinivasan.surprise@gmail.com>

update: ysql latest driver and the default image tag

b247c56

Signed-off-by: Srinivasa Vasu <srinivasan.surprise@gmail.com>

update: build info

ab9773e

Signed-off-by: Srinivasa Vasu <srinivasan.surprise@gmail.com>

version update to 2.12

cb4f3b0

Signed-off-by: Srinivasa Vasu <srinivasan.surprise@gmail.com>

version update to 2.12 for the test

ad5f960

Signed-off-by: Srinivasa Vasu <srinivasan.surprise@gmail.com>

refactor from yugabyte to yugabytedb

e645fb3

Signed-off-by: Srinivasa Vasu <srinivasan.surprise@gmail.com>

add incubating module note to the module readme file

415eef3

Signed-off-by: Srinivasa Vasu <srinivasan.surprise@gmail.com>

update dependency info in doc

38c74ee

Signed-off-by: Srinivasa Vasu <srinivasan.surprise@gmail.com>

update doc alignment issue

2272f85

Signed-off-by: Srinivasa Vasu <srinivasan.surprise@gmail.com>

update gradle build file

41b4a17

Signed-off-by: Srinivasa Vasu <srinivasan.surprise@gmail.com>

rename test classes

058527a

Signed-off-by: Srinivasa Vasu <srinivasan.surprise@gmail.com>

update readme

f17bc5e

Signed-off-by: Srinivasa Vasu <srinivasan.surprise@gmail.com>

add yugabytedb to the .github template files for issue/enhancement tr…

112f4eb

…acking Signed-off-by: Srinivasa Vasu <srinivasan.surprise@gmail.com>

update assertj and removed cassandra driver dependency

4af7124

Signed-off-by: Srinivasa Vasu <srinivasan.surprise@gmail.com>

cosmetic fix: checkstyleTest and spotlessApply

9329e83

Signed-off-by: Srinivasa Vasu <srinivasan.surprise@gmail.com>

update yugabytedb labeler

57d170c

Signed-off-by: Srinivasa Vasu <srinivasan.surprise@gmail.com>

update yugabytedb tag

1bc628e

Signed-off-by: Srinivasa Vasu <srinivasan.surprise@gmail.com>

srinivasa-vasu dismissed eddumelendez’s stale review via 1bc628e November 11, 2022 10:46

srinivasa-vasu force-pushed the yugabyte branch from 0af5d61 to 1bc628e Compare November 11, 2022 10:46

Polish

4cee74e

eddumelendez added this to the next milestone Nov 15, 2022

eddumelendez merged commit 858dd0a into testcontainers:main Nov 15, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add YugabyteDB module #4372

Add YugabyteDB module #4372

srinivasa-vasu commented Aug 15, 2021 •

edited

nchandrappa commented Dec 3, 2021

paskos commented Dec 14, 2021

srinivasa-vasu commented Apr 27, 2022

eddumelendez left a comment

srinivasa-vasu commented Jul 26, 2022

eddumelendez commented Jul 26, 2022

eddumelendez left a comment

eddumelendez Aug 1, 2022

srinivasa-vasu Oct 23, 2022

eddumelendez commented Aug 4, 2022 •

edited

srinivasa-vasu commented Aug 31, 2022

eddumelendez left a comment

eddumelendez Oct 17, 2022

kiview Oct 18, 2022

srinivasa-vasu Oct 23, 2022

eddumelendez left a comment

eddumelendez Oct 24, 2022

eddumelendez commented Nov 15, 2022

srinivasa-vasu commented Nov 16, 2022

Add YugabyteDB module #4372

Add YugabyteDB module #4372

Conversation

srinivasa-vasu commented Aug 15, 2021 • edited

nchandrappa commented Dec 3, 2021

paskos commented Dec 14, 2021

srinivasa-vasu commented Apr 27, 2022

eddumelendez left a comment

Choose a reason for hiding this comment

srinivasa-vasu commented Jul 26, 2022

eddumelendez commented Jul 26, 2022

eddumelendez left a comment

Choose a reason for hiding this comment

eddumelendez Aug 1, 2022

Choose a reason for hiding this comment

srinivasa-vasu Oct 23, 2022

Choose a reason for hiding this comment

eddumelendez commented Aug 4, 2022 • edited

srinivasa-vasu commented Aug 31, 2022

eddumelendez left a comment

Choose a reason for hiding this comment

eddumelendez Oct 17, 2022

Choose a reason for hiding this comment

kiview Oct 18, 2022

Choose a reason for hiding this comment

srinivasa-vasu Oct 23, 2022

Choose a reason for hiding this comment

eddumelendez left a comment

Choose a reason for hiding this comment

eddumelendez Oct 24, 2022

Choose a reason for hiding this comment

eddumelendez commented Nov 15, 2022

srinivasa-vasu commented Nov 16, 2022

srinivasa-vasu commented Aug 15, 2021 •

edited

eddumelendez commented Aug 4, 2022 •

edited