How to output custom json log message

[missedone]

hi folks
i'd like to output the keycloak logs in the custom json format, i know keycloak uses quarkus which use jboss-logging, but the problem with jboss-logging is it doesn't support escape quotes or encode multiline string in the json message https://www.keycloak.org/server/logging#_configuring_the_console_log_format

how can i make keycloak use log4j2? or do you see any other option?

thanks

[codespearhead]

Are you saying that KC_LOG_CONSOLE_OUTPUT=json does not work?

[missedone]

hi, KC_LOG_CONSOLE_OUTPUT=json gives the default json format while i want to custom the json message format

[missedone]

in my case, i'd like to get the log output in format of {"applog":{"EventTime":"%d{yyyy-MM-dd HH:mm:ss,SSS}","meta":{"level":"%-5p","logger":"%c","thread":"%t"},"event":"%s"}}

[codespearhead]

I see. If a custom format via KC_LOG_CONSOLE_FORMAT as well as using Quarkus directly does not work, the the only way out I can think of is via file log parsing.

[missedone]

I tried KC_LOG_CONSOLE_FORMAT with the above json format, but the log didn’t show up at all on console, I guess it could be because of the braces, the simple plain format without braces worked

[codespearhead]

Try it like this:

docker run -p 8080:8080 \
-e KEYCLOAK_ADMIN=admin \
-e KC_LOG_CONSOLE_OUTPUT=json \
-e KEYCLOAK_ADMIN_PASSWORD=admin \
-e KC_LOG_CONSOLE_FORMAT='{"applog":{"EventTime":"%d{yyyy-MM-dd HH:mm:ss,SSS}","meta":{"level":"%-5p","logger":"%c","thread":"%t"},"event":"%s"}}' \
quay.io/keycloak/keycloak:24.0.3 start-dev

It seems to be working on my end.

[missedone]

@codespearhead , interesting, it worked now with KC_LOG_CONSOLE_FORMAT, not sure what wrong before.

thanks

[missedone]

hmm, with more test, the solution is not perfect because it can't handle/escape the double quotes, new lines and other special characters in the log message.
Ex. below is not a valid json:

{"applog":{"EventTime":"2024-04-29 21:06:25,525","meta":{"level":"WARN ","logger":"org.keycloak.events","thread":"executor-thread-94"},"event":"type="INTROSPECT_TOKEN_ERROR", realmId="5384fc72-1822-438d-96ee-a851b3f93e5f", clientId="example-svc", userId="null", ipAddress="100.126.62.50", error="token_introspection_failed", reason="Could not find valid user session for session_state = 4b57f130-529a-4b2d-bde6-c09fa7927b47", client_auth_method="client-secret""}}

[codespearhead]

What action did you perform on the server to get the above output?

Either way, I updated my previous answer to add the missing KC_LOG_CONSOLE_OUTPUT=json. I suggest trying it again.

[missedone]

@codespearhead thanks for the update, if KC_LOG_CONSOLE_OUTPUT=json is set, it use the default json format rather than my custom json format defined in KC_LOG_CONSOLE_FORMAT

[missedone]

anyway, i will take your suggestion as it's the most viable option.

at the server side, we have filebeat collect the logs, so i will have dedicate log parse pipeline in filebeat/ES for keycloak.