Dependency management for Apache Kafka is incomplete

[artembilan]

The current Apache Kafka management is like this:

	library("Kafka", "3.0.0") {
		group("org.apache.kafka") {
			modules = [
				"connect-api",
				"connect-basic-auth-extension",
				"connect-file",
				"connect-json",
				"connect-runtime",
				"connect-transforms",
				"kafka-clients",
				"kafka-metadata",
				"kafka-log4j-appender",
				"kafka-streams",
				"kafka-streams-scala_2.12",
				"kafka-streams-scala_2.13",
				"kafka-streams-test-utils",
				"kafka-tools",
				"kafka_2.12",
				"kafka_2.13"
			]
		}
	}

There are two more dependencies which would really use for testing in Spring for Apache Kafka when our @EmbeddedKafka is involved:

        <dependency>
            <groupId>org.apache.kafka</groupId>
            <artifactId>kafka-clients</artifactId>
            <classifier>test</classifier>
            <scope>test</scope>
            <version>${kafka.version}</version>
        </dependency>

        <dependency>
            <groupId>org.apache.kafka</groupId>
            <artifactId>kafka_2.13</artifactId>
            <classifier>test</classifier>
            <scope>test</scope>
            <version>${kafka.version}</version>
        </dependency>

Pay attention to that <classifier>test</classifier>.

They come as transitive dependencies from spring-kafka-test but if we need to override an Apache Kafka version for some reason, we also have to add those two dependencies explicitly: https://docs.spring.io/spring-kafka/docs/2.8.1-SNAPSHOT/reference/html/#update-deps.

It probably would be much nicer just be let end-users to specify a Kafka version like ext['kafka.version'] = '2.8.1' and everything else comes to us from Spring Boot management.

Thanks

P.S. I would PR the fix, but I don't know how specify a test classifier in Gradle configuration 😄

[artembilan]

An Apache Kafka JIRA ticket for BOM publishing request: https://issues.apache.org/jira/browse/KAFKA-9060

[wilkinsona]

This is blocked by #29298.

[garyrussell]

Sweet! Thanks @wilkinsona