NUL-corrupted keys, values on JSON serialization

[jaredstehler]

First posted in discussions: FasterXML/jackson#215

Jackson version: 2.12.6

We have observed sporadic instances of serialized JSON messages containing corrupted keys and/or values (correct length, but some amount of string suffix is replaced with NUL (0) bytes).

From our metrics, it seemed to recur every 2-4 days on average. We have swapped in a fork of jackson-core with fields marked as volatile in SerializedString (HubSpot@4d3bd2d) and have not seen the issue in > 10 days now.

Even though we saw this once or twice a while back on JDK 17, it seems to have been exacerbated with JDK 21. We are also running on ARM64 instances, which could have some behavior contributing here.

I have not been able to reproduce this in isolation, unfortunately. Happy to help provide any additional information. We are working on an upgrade to jackson 2.18 but I see those fields are unchanged in that branch as well.

[pjfanning]

Can you provide us with your code? Ideally as a PR but paste it here if a PR is not something you have time for.

PS We are unlikely to release a new v2.12 fix. The fix is likely to only go in more recent release lines - 2.17 and possibly 2.16, etc.

[jaredstehler]

Oh I wasn't planning on pushing for a 2.12 release; we are definitely moving to a higher version. I only called it out as a clarifying point of context.

[jaredstehler]

Which code are you looking for? The actual invocations to the ObjectMapper? Types being serialized?

[pjfanning]

Which code are you looking for? The actual invocations to the ObjectMapper? Types being serialized?

I was asking for a PR with the changes you want to SerializedString.

[jaredstehler]

ah, sorry. opened PR #1276