Build fails for common-config

[anshumandeb]

running mvn -pl :common-config package from the parent folder gives the following error:

[ERROR] Failed to execute goal on project common-config: Could not resolve dependencies for project io.confluent:common-config:jar:5.2.0-SNAPSHOT: The following artifacts could not be resolved: io.confluent:common-utils:jar:5.2.0-SNAPSHOT, io.confluent:assembly-plugin-boilerplate:zip:resources:5.2.0-SNAPSHOT: Failure to find io.confluent:common-utils:jar:5.2.0-SNAPSHOT in http://packages.confluent.io/maven/ was cached in the local repository, resolution will not be reattempted until the update interval of confluent has elapsed or updates are forced -> [Help 1]

[hashhar]

Any updates on this? This is impeding work on confluentinc/kafka-connect-jdbc#401.

[RoyalFeng]

can some one slove this problem？ I have met same issue.

[OneCricketeer]

@RoyalFeng It's not clear what specific error you are getting, but you will need to build Kafka source code before you can build this repo from the master branch.

• Kafka - clone https://github.com/confluentinc/kafka.git and build with ./gradlew installAll
• Common - clone https://github.com/confluentinc/common and build with mvn install

[ewencp]

There's nothing broken here, you are trying to build a single module without building its parent modules, and those SNAPSHOTS aren't going to be published in maven central or Confluent's production release maven repository. You should build the full project. If you are just trying to develop downstream projects, you can mvn install from the root of this project.

[daxiangxiaoyi]

• https://github.com/confluentinc/kafka.git

I followed your steps,but still failed.

[hashhar]

@daxiangxiaoyi If you can afford to, please clear your local maven repo or at least move it.

mv ~/.m2 ~/.m2.old
git clone https://github.com/confluentinc/kafka.git
cd kafka
./gradlew installAll
git clone https://github.com/confluentinc/common.git
cd common
mvn install

The above works for me everytime.

[dnascimb]

This works with kafka project on trunk and common project on master branch. If however, I switch the common project to branch 5.2.x, it fails as it is unable to find the snapshots. Are there other conditions that need to be met to make this work?

Here is the failure when running mvn install from common project when on branch 5.2.x after successful installation of kafka project:

[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  20.691 s
[INFO] Finished at: 2019-03-11T12:37:59-07:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal on project common-logging: Could not resolve dependencies for project io.confluent:common-logging:jar:5.2.0-SNAPSHOT: The following artifacts could not be resolved: org.apache.kafka:connect-api:jar:2.2.0-SNAPSHOT, org.apache.kafka:connect-json:jar:2.2.0-SNAPSHOT: Failure to find org.apache.kafka:connect-api:jar:2.2.0-SNAPSHOT in http://packages.confluent.io/maven/ was cached in the local repository, resolution will not be reattempted until the update interval of confluent has elapsed or updates are forced -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/DependencyResolutionException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn <goals> -rf :common-logging

Do I need to switch the kafka project to an alternate branch, and if so, how do I determine what branch is compatible with the branch I have checked out from common?

[ewencp]

You need to use the corresponding Kafka branch for the release. The mapping is documented here. So for example if you want to build the 5.1.x branch you would build the 2.1 branch of Kafka.

[OneCricketeer]

Also, note, the real reason for the failure would be above the [INFO] BUILD FAILURE line. Without that information, it's difficult to tell what the solution could be.

[dnascimb]

Thanks @ewencp and @Cricket007 👍

[imranrajjad]

so what kafka version is required for 5.3.0-SNAPSHOT?

[ewencp]

2.3-SNAPSHOT. You can find the kafka version, regardless of branch, here https://github.com/confluentinc/common/blob/master/pom.xml#L62 (and it'll also be reported in the error message if you try to build without having the dependency installed).

[OneCricketeer]

And you can find the version being built of the Kafka repo here - https://github.com/confluentinc/kafka/blob/trunk/gradle.properties#L23

[MohammadChalaki98]

@daxiangxiaoyi If you can afford to, please clear your local maven repo or at least move it.

mv ~/.m2 ~/.m2.old
git clone https://github.com/confluentinc/kafka.git
cd kafka
./gradlew installAll
git clone https://github.com/confluentinc/common.git
cd common
mvn install

The above works for me everytime.

Using ./gradlew installAll I get the error:
Task 'installAll' not found in root project 'kafka'. Some candidates are: 'install'.
Any idea ?!
Thnx

[OneCricketeer]

Try reading the README

https://github.com/confluentinc/kafka#installing-the-jars-to-the-local-maven-repository

[MohammadChalaki98]

Try reading the README

https://github.com/confluentinc/kafka#installing-the-jars-to-the-local-maven-repository

I've done everything as supposed, still getting error
The POM for io.confluent:resolver-maven-plugin:jar:0.4.0 is missing, no dependency information available

[OneCricketeer]

Error suggests you'll need to install this now

https://github.com/confluentinc/resolver-maven-plugin

[ewencp]

That's correct, this is a very recent change in requirements. I think we're considering just pushing that plugin to maven central so it's easier to get to.

[MohammadChalaki98]

That's correct, this is a very recent change in requirements. I think we're considering just pushing that plugin to maven central so it's easier to get to.

It will be great, since https://github.com/confluentinc/resolver-maven-plugin is for 5 years ago and trying it didnt help with the issue.

[OneCricketeer]

The version you're getting an error for is 9 days old

[MohammadChalaki98]

The version you're getting an error for is 9 days old

Sorry for the bother, I've cloned the resolver package and tried to mvn it. But I'm still getting the same error:
Plugin io.confluent:resolver-maven-plugin:0.4.0 or one of its dependencies could not be resolved: Could not find artifact io.confluent:resolver-maven-plugin:jar:0.4.0 in confluent

Could you please explain if I had to deploy the resolver package by any special command or get it in any special directory or something?! THNX!

[OneCricketeer]

You'll need to checkout the tag for the version in the error first
https://github.com/confluentinc/resolver-maven-plugin/blob/master/pom.xml#L7

[improved-broccoli]

I encounter troubles too to install JAR in my local Maven:

1. I cloned confluentinc/resolver-maven-plugin, checkout tag 0.4.0, ran mvn clean install and it succeed
2. I cloned confluentinc/kafka repo, checkout branch v6.1.0-beta200907192126 and ran ./gradlewAll install and it succeed
3. I cloned confluentinc/common repo, checkout branch v6.1.0-beta200907192126, ran mvn clean install and it failed with:

[ERROR] Failed to execute goal io.confluent:resolver-maven-plugin:0.4.0:resolve-kafka-range (default) on project common-parent: : MojoExecutionException: No matching ce-kafka version found for constraint: 'org.apache.kafka:kafka-clients::[6.1.0-0, 6.1.1-0), excluding snapshots'. -> [Help 1]

What's wrong?
Is there somewhere a complete and up-to-date manual about how to build confluent projects? My goal is to build latest schema-registry version to try a patch not yet published on Confluent Maven.

Thanks.

EDIT: After finding issues here and there, I finally found out how to build JAR against upcoming 6.0:

git clone https://github.com/confluentinc/resolver-maven-plugin.git
git checkout 0.4.x
mvn clean install
cd ..
git clone https://github.com/confluentinc/kafka.git
git checkout 2.6
./gradlewAll install #This must be "./gradlew installAll" if kafka version < 2.6
cd ..
git clone https://github.com/confluentinc/common.git
git checkout 6.0.x
mvn clean install
cd ..
git clone https://github.com/confluentinc/rest-utils.git
git checkout 6.0.x
mvn clean install # One test failed so I had to add -DskipTests
cd ..
git clone https://github.com/confluentinc/schema-registry.git
git checkout 6.0.x
mvn clean install

Hope this helps.
By the way, those both FAQ (here and here) should be merged into a single one up-to-date because commands they give does not work against 6.x release.
Thanks.

[OneCricketeer]

If you want to "build latest", for active development, then use master branches

[ldkhanh]

Add this into your pom file

<repositories>
      <repository>
          <id>confluent</id>
          <url>https://packages.confluent.io/maven/</url>
      </repository>
</repositories>

<dependency>
        <groupId>io.confluent</groupId>
        <artifactId>common-utils</artifactId>
        <version>6.0.0</version>
</dependency>

[raphaelauv]

also you can just checkout a release version

like

git checkout v6.0.1