Could not deserialize ATN with version 3 (expected 4) with Hibernate 6

[praval-123]

We are trying to use Apache spark in our existing application. Spark is using Antlr4 version 4.9. While Hibernate 6 is using Antlr4 version 4.10.

Antlr has made changes between 4.9 version and 4.10 version which is causing below issue.

Caused by: java.lang.UnsupportedOperationException: java.io.InvalidClassException: org.antlr.v4.runtime.atn.ATN; Could not deserialize ATN with version 3 (expected 4).

We are using below dependencies in our application which is running with Spring Boot 3.2.3 and Java 17. Below are the maven dependencies we are using.

 <dependency>
                <groupId>org.apache.spark</groupId>
                <artifactId>spark-core_2.12</artifactId>
                <version>3.5.0</version>
            </dependency>
    
            <dependency>
                <groupId>org.apache.spark</groupId>
                <artifactId>spark-sql_2.12</artifactId>
                <version>3.5.0</version>
            </dependency>

          <dependency>
            <groupId>org.hibernate.orm</groupId>
            <artifactId>hibernate-core</artifactId>
            <version>6.2.1.Final</version>
            <exclusions>
                <exclusion>
                    <artifactId>dom4j</artifactId>
                    <groupId>org.dom4j</groupId>
                </exclusion>
            </exclusions>
        </dependency>

[KvanTTT]

It's a duplicate. Please take a look at the following issues:

• Could not deserialize ATN with version 4 (expected 3). #3895
• Could not deserialize ATN with version (expected 4). #3997
• ANTLR Tool version 4.4 used for code generation does not match the current runtime version 4.11.1 #4100

[praval-123]

From above issues, I understand that between antlr 4.9 and 4.10 antlr4 store the ATN serialized in different formats, version 3 and 4. the respective runtimes, coming with the runtime dependency, are incompatible. It means apache spark (latest version) is incompatible with Hibernate 6 which is used with latest Spring boot and Java 17.

Is there any workaround or Apache Spark needs to updated with antlr4 4.10 version only?

[ericvergnaud]

Spark parser needs to be regenerated.