Liquibase 4.3.0 throws NullPointerExceptions without snakeyaml in the classpath

[nvoxland]

Description

Snakeyaml is the 3rd party library we use for JSON and YAML parsing.

It has always been an optional component, traditionally only needed for json/yaml changelogs and offline snapshots (which are json-formatted).

As we are adding hub.liquibase.com based functionality, that also requires snakeyaml for the json rest API calls.

With 4.3.0, there is an issue where the "get the HubService" implementation call returns null if we cannot instantiate the standard hubservice class, and then the code that expects a service to be returned throws the null pointer exception.

Impact

This should NOT impact:

• CLI users (because snakeyaml is included in the CLI distribution)
• Liquibase-Maven-Plugin users, because Maven appears to include snakeyaml in the liquibase classpath with the <optional>true</optional> setting we have in the plugin's pom.xml
• Any spring boot or other embedded users, where snakeyaml is not explicitly included as a dependency

This SHOULD impact:

• Any spring boot or other embedded setups, where snakeyaml is not explicitly included as a dependency. We have it listed as true in our pom.xml, so it should not be automatically included
• Ant users
• Anyone else running liquibase without snakeyaml.jar in their classpath.

Resolution Options

1. We can make snakeyaml a required dependency. As we continue to add hub-based functionality, the ability to use json will become more and more central that it's not worth keeping it as an optional dependency.
2. Continue to keep snakeyaml optional, and improve test coverage around how to handle cases where the library is not available

NOTE: Using a different library than snakeyaml for json parsing is a different issue.

Liquibase Internal QA Test Requirements

• Verify Liquibase CLI update does not throw a null pointer if snakeyaml is not in path.
• Verify Liquibase Ant update does not throw a null pointer if snakeyaml is not in the path.
• Verify Liquibase Servlet update does not throw a null pointer if snakeyaml is not in the path.
• Verify Liquibase Maven update does not throw a null pointer.
  • Fix should have no impact on Maven for reasons described in the Impact section.
• Verify all automated functional CLI tests pass on Jenkins.

Jenkins is an internal build server and results are not visible to the wider community.

[nvoxland]

As an embeddable library, we've always tried to keep 3rd party dependencies to a minimum (ideally zero). That makes me not want to make snakeyaml required, even as more of our functionality will depend on it.

We have a work-around fix for HubService returning null, so that is probably the best short-term fix anyway. Then we can consider the longer-term plans around how to balance improved json support with additional dependencies.

[kdebski85]

I also encountered the issue with 4.3.0.
Here is the stacktrace for reference:

liquibase.exception.LiquibaseException: java.lang.NullPointerException                             
        at liquibase.Liquibase.runInScope(Liquibase.java:2348)                                     
        at liquibase.Liquibase.update(Liquibase.java:216)                                          
        at liquibase.Liquibase.update(Liquibase.java:202)                                          
        at liquibase.integration.commandline.Main.doMigration(Main.java:1764)                      
        at liquibase.integration.commandline.Main$1.lambda$run$0(Main.java:364)                    
        at liquibase.integration.commandline.Main$1$$Lambda$3.0000000010621090.run(Unknown Source) 
        at liquibase.Scope.lambda$child$0(Scope.java:160)                                          
        at liquibase.Scope$$Lambda$4.0000000010CACB20.run(Unknown Source)                          
        at liquibase.Scope.child(Scope.java:169)                                                   
        at liquibase.Scope.child(Scope.java:159)                                                   
        at liquibase.Scope.child(Scope.java:138)                                                   
        at liquibase.integration.commandline.Main$1.run(Main.java:363)                             
        at liquibase.integration.commandline.Main$1.run(Main.java:193)                             
        at liquibase.Scope.child(Scope.java:169)                                                   
        at liquibase.Scope.child(Scope.java:145)                                                   
        at liquibase.integration.commandline.Main.run(Main.java:193)                               
        at liquibase.integration.commandline.Main.main(Main.java:156)                              
Caused by: java.lang.NullPointerException                                                          
        at liquibase.hub.HubUpdater.register(HubUpdater.java:306)                                  
        at liquibase.Liquibase.lambda$update$1(Liquibase.java:242)                                 
        at liquibase.Liquibase$$Lambda$8.0000000010CA73E0.run(Unknown Source)                      
        at liquibase.Scope.lambda$child$0(Scope.java:160)                                          
        at liquibase.Scope$$Lambda$4.0000000010CACB20.run(Unknown Source)                          
        at liquibase.Scope.child(Scope.java:169)                                                   
        at liquibase.Scope.child(Scope.java:159)                                                   
        at liquibase.Scope.child(Scope.java:138)                                                   
        at liquibase.Liquibase.runInScope(Liquibase.java:2343)                                     
        ... 16 more

[nvoxland]

Yes, that is the issue.

What were you using to run Liquibase?

[mthmulders]

I'm not using Liquibase Hub (as far as I know, I've never registered for it). Do I understand correctly that Liquibase nevertheless tries to connect to a remote service?

[nvoxland]

No it does not unless you explicitly configure Liquibase to connect. The null pointer exception happens in the code that is checking to see if you have that local setting enabled as a first step.

[mthmulders]

No it does not unless you explicitly configure Liquibase to connect. The null pointer exception happens in the code that is checking to see if you have that local setting enabled as a first step.

Thanks for the reassurance!

[sync-by-unito]

➤ Erzsebet Carmean commented:

Liquibase 4.3.1-LB-1212-SNAPSHOT #1Hub {"git":{"branch":"staging","commit":{"id":"66f7cf2","time":"2021-01-28T19:25:33Z"}}}Liquibase Internal QA Test Requirements

• Verify Liquibase CLI update does not throw a null pointer if snakeyaml is not in path. PASS

  • When executing with java -jar PASS
  • When executing with standard Liquibase command PASS

• Verify Liquibase CLI dropAll does not throw a null pointer if snakeyaml is not in path. PASS

  • When executing with java -jar PASS
  • When executing with standard Liquibase command PASS

• Verify Liquibase Ant update does not throw a null pointer if snakeyaml is not in the path. PASS

• Verify Liquibase Servlet update does not throw a null pointer if snakeyaml is not in the path. PASS

• Verify fix has no impact on Liquibase Maven update PASS

• Verify fix has no impact on Liquibase Maven dropAll PASS

• Verify all automated functional CLI tests pass on Jenkins. PASS (LB-1212, Build Permit arbitrary attributes from foreign namespaces in some elements of XML changelogs #2)

Everything here looks good.