Improvements to oj/json?

[ioquatix]

Would it be possible to improve the compatibility of JSON parsers/generators and avoid things like multi_json by extending the JSON interface?

I don't know how much we can achieve here, but if we standardised JSON#generator= and JSON#parser= or something like that, we could make pluggable JSON more easy, and less monkey-patchy. I'd be happy to try and propose that for Ruby 3.3+.

I actually don't know the full extent of how compatible those interfaces are across CRuby, TruffleRuby and JRuby, but I assume there is some level of standardisation. Feel free to educate me if you can fill in the gaps.

[ohler55]

The relationships are complex. No doubt about that. The JSON gem and Oj take different approaches to parsing JSON. By using different approaches Oj has been able to hold a performance gain over the JSON gem. The situation is worse still in that Rails introduces yet another set of APIs. Oj in compat more should act just like the JSON gem. If some of the other Oj features are desired or preferred they are available. Trying to force all JSON parsing and output to have exactly the same APIs will only work if one get is picked and the others discarded if for no other reason than the module names will not be the same.

[ohler55]

Expanding on the previous answer lets separate JSON libraries from Ruby implementations. Take Ruby for example, JRuby does not support C extensions so any library making use of C extensions for performance can not be used. A good summary can be found at https://engineering.appfolio.com/appfolio-engineering/2017/12/28/cruby-mri-jruby-rubyspec-rubinius-yarv-a-little-bit-of-ruby-naming.

Now JSON libraries. Parsing is one area where implementations vary mostly in regards to how Ruby objects are created but not in a significant way. The big differences are in the marshalling of Ruby objects into JSON. Two fundamentally different approaches are used.

1. The Ruby objects encode themselves either into a JSON string (#to_json) or into base JSON type (#as_json).

2. A library encodes objects after introspection of those objects.

The JSON gem and Active Support (Ruby on Rails) use the first approach while Oj uses the second approach.

Advantages of approach 1 are that each class of objects can decide how to encode themselves. The disadvantages are that each object decides how to encode themselves which means each class needs to be monkey patched to support the encoding. That works fine until there is more than once way to encode the objects then you get competition over which monkey patch will be used. This was the bane of Rails in the earlier versions. Currently that issue has mostly been resolved by using different methods for the JSON gem and Rails which encourages the use of #as_json as apposed to #to_json that the JSON uses. Another disadvantage is that all encoding has to make a Ruby call instead of optimizing by using the C extension APIs for encoding.

The advantages of approach 2 is that the library can decide how to encode objects which allows for encoding into different structured JSON. This avoids having to monkey patch classes and risk collisions with other monkey patches. It also allows staying primarily in C for optimization of performance. Custom class based encoding is still possible with a specific method such as #my_json or something more aptly named.

Multi_json does a good job of providing a common API even if it is a subset of the features of each library. Unless a common set of APIs similar to multi_json can be agreed on and added to all libraries, multi_json is the current best alternative.