Upgrade owasp-java-html-sanitizer from 20220608.1 to 20240325.1

[casewalker]

Closes #28385

There may be a direct vulnerability from this dependency (CVE-2011-4457), but removing a transitive dependency on Guava is at least a step in the right direction.

As well, removing Guava (and also apparently some FindBugs dependency) removed the availability of:

• com.google.common.base.Strings#isNullOrEmpty
• com.google.common.base.Charsets#UTF_8
• com.google.common.base.Predicate
• org.checkerframework.checker.nullness.qual.NonNull
• com.google.common.collect.(ImmutableMap,ImmutableList,ImmutableSet)

These were all replaced with:

• org.keycloak.utils.StringUtil#isNullOrEmpty
• java.nio.charset.StandardCharsets#UTF_8
• java.util.function.Predicate
• jakarta.annotation.Nonnull
• java.util.Collections

Respectively in a variety of places throughout the code.

Beyond that, this upgrade required new dependencies from owasp-java-html-sanitizer, some new "Java Shim" dependencies added to that project, so those dependencies were added here too wherever necessary.

[mposolda]

@casewalker Thanks for the PR! Are you please able to add the signature to the commit message as other commits in Keycloak repository? See https://github.com/keycloak/keycloak/blob/main/CONTRIBUTING.md#developers-certificate-of-origin for the details.

@pskopek Do you think it is ok to update this dependency?

[casewalker]

@mposolda Added the signature now.

[mposolda]

@casewalker Thanks. It seems there are compilation failures in the services modules. It seems it is caused by your change. Could you please doublecheck?

[casewalker]

It seems there were some sporadic Guava dependencies still throughout the repo, only one was directly related to owasp-java-html-sanitizer (in the services class org.keycloak.theme.KeycloakSanitizerPolicy), the rest were uses of either com.google.common.base.Strings#isNullOrEmpty or com.google.common.base.Charsets#UTF_8. These were all easily replaced with org.keycloak.utils.StringUtil and java.nio.charset.StandardCharsets respectively.

For the class KeycloakSanitizerPolicy, it says it is based on the example code in EbayPolicyExample.java, which appears to have switched from com.google.common.base.Predicate to java.util.function.Predicate, so I did the same in KeycloakSanitizerPolicy.

With the Guava issues addressed, the services were still failing to compile with:

keycloak/services/src/main/java/org/keycloak/credential/WebAuthnCredentialProvider.java:[36,50] package org.checkerframework.checker.nullness.qual does not exist

So I switched out the @NonNull annotation from org.checkerframework.checker.nullness.qual.NonNull to the jakarta.annotation.Nonnull annotation already used elsewhere. I am not sure how that checkerframework dependency is related here, but it was an easy enough fix.

This PR is now bigger than it was, please let me know your thoughts!

[casewalker]

Test failing with:

AppInitiatedActionWebAuthnTest ++ 2024-04-09 16:14:19,148 ERROR [org.keycloak.services.error.KeycloakErrorHandler] (executor-thread-2) Uncaught server error: java.lang.NoClassDefFoundError: Could not initialize class org.keycloak.theme.KeycloakSanitizerPolicy
AppInitiatedActionWebAuthnTest ++   at org.keycloak.theme.KeycloakSanitizerMethod.exec(KeycloakSanitizerMethod.java:47)
...
AppInitiatedActionWebAuthnTest ++ Caused by: java.lang.ExceptionInInitializerError: Exception java.lang.NoClassDefFoundError: org/owasp/shim/Java8Shim [in thread "executor-thread-3"]
AppInitiatedActionWebAuthnTest ++   at org.owasp.html.HtmlPolicyBuilder.<clinit>(HtmlPolicyBuilder.java:168)
AppInitiatedActionWebAuthnTest ++   at org.keycloak.theme.KeycloakSanitizerPolicy.<clinit>(KeycloakSanitizerPolicy.java:84)
AppInitiatedActionWebAuthnTest ++   ... 43 more

I wasn't paying quite enough attention on the OWASP dependency upgrade, it seems it now also needs this Java8Shim dependency. It looks like in issue OWASP/java-html-sanitizer#329, they suggest adding multiple shims to the dependencies and letting the right one get selected at runtime. I have now done that.

[casewalker]

I made some branch mistakes trying to update from main and add the dependencies, but it should be all settled now.

[casewalker]

I removed this since I also replaced the no-longer-existent @NonNull with the one from Jakarta.

[casewalker]

Not sure if exclusions are necessary, they were done above so I just did them here too. Things seemed to work out ok with this.

[casewalker]

It's been a while since I really worked on Java and Maven. I forgot to pull the dependency in on the keycloak-services pom.xml and other places the dependency is used. Sorry about that.

This time, I ran the code in the IDELauncher just to see that it can startup successfully. I ran it before changing the poms and saw the server error, then updated the poms and got through without errors.

[casewalker]

I am not sure what happened in Keycloak Operator CI / Test local, it looks like maybe it just timed out?

INFO: keycloak-operator stopped in 0.024s
[INFO] 
[INFO] Results:
[INFO] 
Error:  Errors: 
Error:    CacheTest.testCacheConfigMapFile:82 » ConditionTimeout Assertion condition defined as a org.keycloak.operator.testsuite.utils.K8sUtils 
expected: true
 but was: false within 5 minutes.
[INFO] 
Error:  Tests run: 117, Failures: 0, Errors: 1, Skipped: 0

Does anyone know what might have happened here? Should we retry that test to see if it was just taking longer than usual? Or was this an issue with my code change?

[casewalker]

Hey @mposolda, please let me know if there is anything I could/should do to try to get this PR landed.

[mposolda]

@casewalker Thanks, we need to verify internally if we can update dependencies as every dependency update should be carefully in our internal productization. So added also @pskopek to review this PR, who can provider more details.

One quick question: Keycloak server needs to support just java 17 and newer. So considering this, is it really needed to have both java8-shim and java10-shim in the dependencies? Isn't it just java10-shim sufficient (or ideally none of them, but I guess this is not possible?)

[ssilvert]

@casewalker Why did you request a review from the UI team? I don't see any UI code that is affected.

[mposolda]

@ssilvert I guess it might be accident with "branch mistakes" as @casewalker mentioned above when he probably added some unrelated commits to this PR, which did some changes in the UI? I am removing the team/ui label from this.

[casewalker]

@mposolda Thanks for keeping this PR in order. Yes, the UI team was pulled in when I screwed up merging updates from main, thank you for dropping that review team.

As for the shim dependencies, they are new jars released as sub-modules of the same java-html-sanitizer repo, they seem to allow that code to work in Java10+ as well as the lower versions down to 8. I am not sure exactly why they were added in this way, and I agree that it doesn't seem like Keycloak should need the Java8 Shim, but the Java10 Shim lists it as a dependency, so I was thinking dependencies might break if you only include the Java10 Shim. I also was following the suggestion made here for reference.

Deeper questions on the topic may be out of my depth, but please let me know if there are more questions or concerns.

[pskopek]

LGTM! We have productized new dependency.

[mposolda]

@pskopek Thanks!

@casewalker Could you please just fix the conflicts? I hope we can have this merged afterwards. Class CustomFuseContainer was removed from the codebase in the meantime, so you can probably remove all changes of this class from your PR.

[casewalker]

It looks like more Guava was used since I opened this PR. It was used just for creating immutable data structures, so I replaced all of them with unmodifiable data structures from the java Collections framework.

[mposolda]

@casewalker @pskopek Thanks to both for all the work on this!