BUG: Error serializing Avro schema containing timestamp logical types

[abhilash-ghosh]

Background

As per Avro specification 1.11.0, logical-types Timestamp (millisecond precision) should be supported. Until kafka-rest version 6.2.14 it's working fine. I found that starting v7, this is broken.

How to reproduce
To demonstrate the issue, use the AVRO schema below:

{
  "type": "record",
  "name": "Key",
  "namespace": "com.example",
  "doc": "Avro schema for Kafka Key",
  "fields": [
    {
      "name": "firstName",
      "type": "string",
      "doc": "First name of the user"
    },
    {
      "name": "lastName",
      "type": "string",
      "doc": "last name of the user"
    },
    {
      "name": "timestamp",
      "type": {
        "type": "long",
        "logicalType": "timestamp-millis"
      },
      "doc": "Timestamp in number of milliseconds from the unix epoch"
    }
  ]
}

Sample Request:

curl --location 'https://kafka-rest.example.com/topics/test-topic' \
--header 'Accept: application/vnd.kafka.v2+json, application/vnd.kafka+json, application/json' \
--header 'Content-Type: application/vnd.kafka.avro.v2+json' \
--data '{
    "value_schema_id": 43,
    "records": [
        {
            "value": {
                "firstName": "Foo",
                "lastName": "Bar",
                "timestamp": 1709703520
            }
        }
    ]
}'

Error:

{
    "error_code": 40801,
    "message": "Error serializing Avro message"
}

[reneveyj]

Having the same issue upgrading from 7.4.0 to 7.5.3. I suspect this has to do with confluentinc/schema-registry#2805.

I think setting 'avro.use.logical.type.converters=true' for the AvroSerializer somewhere might fix it but I'm not sure how to set it as an environment variable so it's taken into account...

[reneveyj]

Having the same issue upgrading from 7.4.0 to 7.5.3. I suspect this has to do with confluentinc/schema-registry#2805.

I think setting 'avro.use.logical.type.converters=true' for the AvroSerializer somewhere might fix it but I'm not sure how to set it as an environment variable so it's taken into account...

I think we can't set 'avro.use.logical.type.converters=true' without this: 176d4b5
But it's only coming in v7.7...

[Fryie]

I'm encountering what I believe to be the same issue when trying to serialise UUIDs.

However, it looks to me as if older versions of kafka-rest are compatible with newer versions of the other tools from the confluent-stack. So pinning kafka-rest to an older version and upgrading the rest of the stack appears to work, at least for my use cases.