Confluent Cloud Kafka disconnects on createTopics with configEntries

[albrechtflo-hg]

Describe the bug

When trying to create a topic with initial configuration (note that this is required in Confluent Cloud e.g. for Cleanup Policy), the Confluent Cloud Kafka Server directly terminates the connection after receiving the request. When I do not transmit any configEntries, the topic creation succeds.

To Reproduce

Test code:

const { Kafka, logLevel} = require('kafkajs');

const kafka = new Kafka({
    clientId: 'kafka-scripting',
    brokers: ['pkc-4r***.europe-west1.gcp.confluent.cloud:9092'],
    ssl: true,
    sasl: {
        mechanism: 'plain',
        username: 'WA2VI***',
        password: '***'
    },
    logLevel: logLevel.DEBUG,
    connectionTimeout: 10_000,
    authenticationTimeout: 10_000
});

async function testCreateTopic() {
    const admin = kafka.admin();

    const topicConfigs = [
        {
            "topic": "test2",
            "numPartitions": 1,
            "replicationFactor": 3,
            // removing the configEntries property here lets the call succeed.
            "configEntries": [
                {
                    "configName": "cleanup.policy",
                    "configValue": "delete"
                }
            ]
        }
    ];

    await admin.connect();
    return admin.createTopics({validateOnly: false, topics: topicConfigs}).finally(() => admin.disconnect());
}

(async () => {
    try {
        await testCreateTopic().then(() => console.log("SUCCESS."));
    } catch (e) {
        console.log("Error: ", e);
    }
})();

Expected behavior

The topic is created with the desired initial configuration.

Observed behavior

The Confluent Cloud side terminates the connection directly, obviously due to some kind of very bad request:

...
{"level":"DEBUG","timestamp":"2021-11-18T16:44:26.833Z","logger":"kafkajs","message":"[SASLPlainAuthenticator] SASL PLAIN authentication successful","broker":"b24-pkc-4r***.europe-west1.gcp.confluent.cloud:9092"}
{"level":"DEBUG","timestamp":"2021-11-18T16:44:26.845Z","logger":"kafkajs","message":"[Connection] Request CreateTopics(key: 19, version: 3)","broker":"b24-pkc-4r***.europe-west1.gcp.confluent.cloud:9092","clientId":"kafka-script
ing","correlationId":2,"expectResponse":true,"size":63}
{"level":"DEBUG","timestamp":"2021-11-18T16:44:26.863Z","logger":"kafkajs","message":"[Connection] Kafka server has closed connection","broker":"b24-pkc-4r***.europe-west1.gcp.confluent.cloud:9092","clientId":"kafka-scripting"}
{"level":"DEBUG","timestamp":"2021-11-18T16:44:26.864Z","logger":"kafkajs","message":"[Connection] disconnecting...","broker":"b24-pkc-4r***.europe-west1.gcp.confluent.cloud:9092","clientId":"kafka-scripting"}
...
Error:  KafkaJSConnectionClosedError: Closed connection
    at TLSSocket.onEnd (C:\***\node_modules\kafkajs\src\network\connection.js:139:13)
    at TLSSocket.emit (node:events:402:35)
    at endReadableNT (node:internal/streams/readable:1343:12)

Environment:

• OS: Windows 10
• KafkaJS version 1.15.0
• Kafka version: Confluent Cloud, should be latest
• NodeJS version 16.13.0

Additional context

Please note that creating a topic with initial configuration on that very cluster is possible

• via KafkaTool 2.0
• via Apache's Java Client Library

So it is most likely not an issue on Confluent Cloud side.

[Nevon]

The reason is that configEntries should have the type { name: string; value:string }[]. See https://kafka.js.org/docs/admin#create-topics. The Typescript types were just object[], which isn't very helpful.

#1309 adds validation for this and updates the types to be more useful.

[cappelaere]

I have the exact same problem with 2.2.3 that I just downloaded.