Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow setting database name and user when using JDBC URL #594

Closed
Closed
Show file tree
Hide file tree
Changes from 13 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
6 changes: 6 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -46,3 +46,9 @@ node_modules/

.gradle/
build/

# Eclipse IDE files
**/.project
**/.classpath
**/.settings
**/bin/
8 changes: 6 additions & 2 deletions .travis.yml
Original file line number Diff line number Diff line change
Expand Up @@ -6,9 +6,13 @@ sudo: required
services:
- docker

before_cache:
- rm -f $HOME/.gradle/caches/modules-2/modules-2.lock
- rm -fr $HOME/.gradle/caches/*/plugin-resolution/
cache:
directories:
- '$HOME/.gradle'
- $HOME/.gradle/caches/
- $HOME/.gradle/wrapper/

install:
- ./gradlew build -x check
Expand Down Expand Up @@ -40,7 +44,7 @@ jobs:
-v "$(pwd)":"$(pwd)" \
-w "$(pwd)" \
openjdk:8-jdk-alpine \
./gradlew testcontainers:test --tests '*GenericContainerRuleTest'
./gradlew --no-daemon testcontainers:test --tests '*GenericContainerRuleTest'

- stage: deploy
sudo: false
Expand Down
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ All notable changes to this project will be documented in this file.
### Changed
- Abstracted and changed database init script functionality to support use of SQL-like scripts with non-JDBC connections. ([\#551](https://github.com/testcontainers/testcontainers-java/pull/551))
- Added `JdbcDatabaseContainer(Future)` constructor. ([\#543](https://github.com/testcontainers/testcontainers-java/issues/543))
- Mark DockerMachineClientProviderStrategy as not persistable ([\#593](https://github.com/testcontainers/testcontainers-java/pull/593))

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please could you update the changelog to describe this change?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have updated Changelog with items that I feel related to this PR. Please take a look and let me know/feel free to modify as you seem right.

## [1.6.0] - 2018-01-28

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,10 @@ protected boolean isApplicable() {
return true;
}

protected boolean isPersistable() {
return true;
}

/**
* @return highest to lowest priority value
*/
Expand Down Expand Up @@ -93,7 +97,10 @@ public static DockerClientProviderStrategy getFirstValidStrategy(List<DockerClie
LOGGER.warn("Can't instantiate a strategy from {}", it, e);
return Stream.empty();
}
}),
})

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is the formatting in this Stream.concat... block just 'idea' being weird? doesnt seem to follow the 4 space indents everywhere else.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This class is not modified and related for this PR, probably upstream has changed and so you see the difference. I would leave it as-is for now.

Copy link
Member

@rnorth rnorth Mar 18, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please let's see if we can tackle this without regular expressions separately. I'd be very happy if we could do that and make this code easier to read.
Response to wrong comment

// Ignore persisted strategy if it's not persistable anymore
.filter(DockerClientProviderStrategy::isPersistable)
.peek(strategy -> LOGGER.info("Loaded {} from ~/.testcontainers.properties, will try it first", strategy.getClass().getName())),
strategies
.stream()
.filter(DockerClientProviderStrategy::isApplicable)
Expand All @@ -104,7 +111,9 @@ public static DockerClientProviderStrategy getFirstValidStrategy(List<DockerClie
strategy.test();
LOGGER.info("Found Docker environment with {}", strategy.getDescription());

TestcontainersConfiguration.getInstance().updateGlobalConfig("docker.client.strategy", strategy.getClass().getName());
if (strategy.isPersistable()) {
TestcontainersConfiguration.getInstance().updateGlobalConfig("docker.client.strategy", strategy.getClass().getName());
}

return Stream.of(strategy);
} catch (Exception | ExceptionInInitializerError | NoClassDefFoundError e) {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,11 @@ protected boolean isApplicable() {
return DockerMachineClient.instance().isInstalled();
}

@Override
protected boolean isPersistable() {
return false;
}

@Override
protected int getPriority() {
return ProxiedUnixSocketClientProviderStrategy.PRIORITY - 10;
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -37,8 +37,8 @@ public static Iterable<Object[]> data() {
return asList(
new Object[][]{
{"jdbc:tc:mysql:5.5.43://hostname/databasename", false, false, false},
{"jdbc:tc:mysql://hostname/databasename?TC_INITSCRIPT=somepath/init_mysql.sql", true, false, false},
{"jdbc:tc:mysql://hostname/databasename?TC_INITFUNCTION=org.testcontainers.jdbc.JDBCDriverTest::sampleInitFunction", true, false, false},
{"jdbc:tc:mysql://hostname/databasename?user=someuser&password=somepwd&TC_INITSCRIPT=somepath/init_mysql.sql", true, false, false},
{"jdbc:tc:mysql://hostname/databasename?user=someuser&password=somepwd&TC_INITFUNCTION=org.testcontainers.jdbc.JDBCDriverTest::sampleInitFunction", true, false, false},
{"jdbc:tc:mysql://hostname/databasename?useUnicode=yes&characterEncoding=utf8", false, true, false},
{"jdbc:tc:mysql://hostname/databasename", false, false, false},
{"jdbc:tc:mysql://hostname/databasename?useSSL=false", false, false, false},
Expand Down Expand Up @@ -100,7 +100,21 @@ private void performTestForScriptedSchema(String jdbcUrl) throws SQLException {
assertEquals("A basic SELECT query succeeds where the schema has been applied from a script", "hello world", resultSetString);
return true;
});


result = new QueryRunner(dataSource).query("select CURRENT_USER()", rs -> {
rs.next();
String resultUser = rs.getString(1);
assertEquals("User from query param is created.", "someuser@%", resultUser);
return true;
});

result = new QueryRunner(dataSource).query("SELECT DATABASE()", rs -> {
rs.next();
String resultDB = rs.getString(1);
assertEquals("Database name from URL String is used.", "databasename", resultDB);
return true;
});

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is just a trivial comment, but please could you reformat using 4 spaces for indentation? IntelliJ/Eclipse defaults ought to be reasonably close. I'll add automatic formatting soon...

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, I'll do that. Thanks for reviewing!

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have updated formatting in recent commit. Thanks!

assertTrue("The database returned a record as expected", result);

}
Expand Down
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
package org.testcontainers.containers;

import org.testcontainers.jdbc.ConnectionUrl;

/**
* Base class for classes that can provide a JDBC container.
*/
Expand All @@ -8,4 +10,14 @@ public abstract class JdbcDatabaseContainerProvider {
public abstract boolean supports(String databaseType);

public abstract JdbcDatabaseContainer newInstance(String tag);

/**
* Get the new Instance with Tag and Url. Default Implementation delegates call to {@link #newInstance(tag)} method.
* @param tag
* @param url
* @return
*/
public JdbcDatabaseContainer newInstance(ConnectionUrl url) {
return newInstance(url.getImageTag());
}
}
208 changes: 208 additions & 0 deletions modules/jdbc/src/main/java/org/testcontainers/jdbc/ConnectionUrl.java
Original file line number Diff line number Diff line change
@@ -0,0 +1,208 @@
package org.testcontainers.jdbc;

import java.util.HashMap;
import java.util.Map;
import java.util.Objects;
import java.util.Optional;
import java.util.StringJoiner;
import java.util.regex.Matcher;
import java.util.regex.Pattern;

import lombok.AllArgsConstructor;
import lombok.Getter;

/**
* This is an Immutable class holding JDBC Connection Url and its parsed components, used by {@link ContainerDatabaseDriver}.
*
* {@link ConnectionUrl#parseUrl()} method must be called after instantiating this class.
*
* @author manikmagar
*
*/
public class ConnectionUrl {

@Getter
private String url;

private String databaseType;

@Getter
private String imageTag = "latest";

private String dbHostString;

@Getter
private boolean inDaemonMode = false;

@Getter
private Optional<String> databaseHost = Optional.empty();

@Getter
private Optional<Integer> databasePort = Optional.empty();

@Getter
private Optional<String> databaseName = Optional.empty();

@Getter
private Optional<String> initScriptPath = Optional.empty();

@Getter
private Optional<InitFunctionDef> initFunction = Optional.empty();

@Getter
private Optional<String> queryString;

private ConnectionUrl() {
//Not Allowed here
}

public ConnectionUrl(final String url) {
this.url = Objects.requireNonNull(url, "Connection URL cannot be null");
}

public String getDatabaseType() {
return Objects.requireNonNull(this.databaseType, "Database Type cannot be null. Have you called parseUrl() method?");
}


/**
* This is a part of the connection string that may specify host:port/databasename.
* It may vary for different clients and so clients can parse it as needed.
* @return
*/
public String getDbHostString() {
return Objects.requireNonNull(this.dbHostString, "Database Host String cannot be null. Have you called parseUrl() method?");
}

public static boolean accepts(final String url) {
return url.startsWith("jdbc:tc:");
}

public void parseUrl() {
/*
Extract from the JDBC connection URL:
* The database type (e.g. mysql, postgresql, ...)
* The docker tag, if provided.
* The URL query string, if provided
*/
Matcher urlMatcher = Patterns.URL_MATCHING_PATTERN.matcher(this.getUrl());
if (!urlMatcher.matches()) {
//Try for Oracle pattern
urlMatcher = Patterns.ORACLE_URL_MATCHING_PATTERN.matcher(this.getUrl());
if(!urlMatcher.matches()) {
throw new IllegalArgumentException("JDBC URL matches jdbc:tc: prefix but the database or tag name could not be identified");
}
}
databaseType = urlMatcher.group(1);

imageTag = Optional.ofNullable(urlMatcher.group(3)).orElse("latest");

//String like hostname:port/database name, which may vary based on target database.
//Clients can further parse it as needed.
dbHostString = urlMatcher.group(4);

//In case it matches to the default pattern
Matcher dbInstanceMatcher = Patterns.DB_INSTANCE_MATCHING_PATTERN.matcher(dbHostString);
if(dbInstanceMatcher.matches()) {
databaseHost = Optional.of(dbInstanceMatcher.group(1));
databasePort = Optional.ofNullable(dbInstanceMatcher.group(3)).map(value -> Integer.valueOf(value));
databaseName = Optional.of(dbInstanceMatcher.group(4));
}

queryString = Optional.ofNullable(urlMatcher.group(5));
getQueryParameters();

Matcher matcher = Patterns.INITSCRIPT_MATCHING_PATTERN.matcher(this.getUrl());
if(matcher.matches()) {
initScriptPath = Optional.ofNullable(matcher.group(2));
}

Matcher funcMatcher = Patterns.INITFUNCTION_MATCHING_PATTERN.matcher(this.getUrl());
if(funcMatcher.matches()) {
initFunction = Optional.of(new InitFunctionDef(funcMatcher.group(2), funcMatcher.group(4)));
}

Matcher daemonMatcher = Patterns.DAEMON_MATCHING_PATTERN.matcher(this.getUrl());
inDaemonMode = daemonMatcher.matches() ? Boolean.parseBoolean(daemonMatcher.group(2)) : false;

}

/**
* Get the TestContainers Parameters such as Init Function, Init Script path etc.
* @return {@link Map}
*/
public Map<String, String> getContainerParameters() {

Map<String, String> results = new HashMap<>();

Matcher matcher = Patterns.TC_PARAM_MATCHING_PATTERN.matcher(this.getUrl());
while (matcher.find()) {
String key = matcher.group(1);
String value = matcher.group(2);
results.put(key, value);
}

return results;
}

/**
* Get all Query paramters specified in the Connection URL after ?. This also includes TestContainers parameters.
* @return {@link Map}
*/
public Map<String, String> getQueryParameters() {

Map<String, String> results = new HashMap<>();
StringJoiner query = new StringJoiner("&");
Matcher matcher = Patterns.QUERY_PARAM_MATCHING_PATTERN.matcher(this.getQueryString().orElse(""));
while (matcher.find()) {
String key = matcher.group(1);
String value = matcher.group(2);
if(!key.startsWith("TC_")) query.add(key + "=" + value);
results.put(key, value);
}

queryString = Optional.of("?" + query.toString());
return results;
}

@Override
public boolean equals(Object obj) {
if(Objects.isNull(obj) || !(obj instanceof ConnectionUrl)) return false;
return this.getUrl().equals(((ConnectionUrl)obj).getUrl());
}

/**
* This interface defines the Regex Patterns used by {@link ConnectionUrl}.
*
* @author manikmagar
*
*/
public interface Patterns {
final Pattern URL_MATCHING_PATTERN = Pattern.compile("jdbc:tc:([a-z]+)(:([^:]+))?://([^\\?]+)(\\?.*)?");

final Pattern ORACLE_URL_MATCHING_PATTERN = Pattern.compile("jdbc:tc:([a-z]+)(:([^(thin:)]+))?:thin:@([^\\?]+)(\\?.*)?");

//Matches to part of string - hostname:port/databasename
final Pattern DB_INSTANCE_MATCHING_PATTERN = Pattern.compile("([^:]+)(:([0-9]+))?/([^\\\\?]+)");

final Pattern DAEMON_MATCHING_PATTERN = Pattern.compile(".*([\\?&]?)TC_DAEMON=([^\\?&]+).*");
final Pattern INITSCRIPT_MATCHING_PATTERN = Pattern.compile(".*([\\?&]?)TC_INITSCRIPT=([^\\?&]+).*");
final Pattern INITFUNCTION_MATCHING_PATTERN = Pattern.compile(".*([\\?&]?)TC_INITFUNCTION=" +
"((\\p{javaJavaIdentifierStart}\\p{javaJavaIdentifierPart}*\\.)*\\p{javaJavaIdentifierStart}\\p{javaJavaIdentifierPart}*)" +
"::" +
"(\\p{javaJavaIdentifierStart}\\p{javaJavaIdentifierPart}*)" +
".*");

final Pattern TC_PARAM_MATCHING_PATTERN = Pattern.compile("(TC_[A-Z_]+)=([^\\?&]+)");

final Pattern QUERY_PARAM_MATCHING_PATTERN = Pattern.compile("([^\\?&=]+)=([^\\?&]+)");

}

@Getter
@AllArgsConstructor
public class InitFunctionDef {
private String className;
private String methodName;
}
}