[Question] How can I use ujson as a Flask encoder/decoder?

[MartinThoma]

See Stack Overflow

What did you do?

from uuid import UUID, uuid4

import ujson as json
from flask import Flask, jsonify
from flask.json import JSONEncoder


class CustomJSONEncoder(JSONEncoder):
    def default(self, obj):
        if isinstance(obj, UUID):
            return str(obj)
        return JSONEncoder.default(self, obj)

    def encode(self, o):
        return json.dumps(o)


app = Flask(__name__)
app.json_encoder = CustomJSONEncoder


@app.route("/")
def index():
    return jsonify({"foo": uuid4()})


app.run()

What did you expect to happen?

When visiting the the page, I expected a dict with a UUID-string

What actually happened?

[2020-10-09 10:54:52,063] ERROR in app: Exception on / [GET]
Traceback (most recent call last):
  File "/home/moose/.pyenv/versions/3.8.3/lib/python3.8/site-packages/flask/app.py", line 2447, in wsgi_app
    response = self.full_dispatch_request()
  File "/home/moose/.pyenv/versions/3.8.3/lib/python3.8/site-packages/flask/app.py", line 1952, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/home/moose/.pyenv/versions/3.8.3/lib/python3.8/site-packages/flask/app.py", line 1821, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/home/moose/.pyenv/versions/3.8.3/lib/python3.8/site-packages/flask/_compat.py", line 39, in reraise
    raise value
  File "/home/moose/.pyenv/versions/3.8.3/lib/python3.8/site-packages/flask/app.py", line 1950, in full_dispatch_request
    rv = self.dispatch_request()
  File "/home/moose/.pyenv/versions/3.8.3/lib/python3.8/site-packages/flask/app.py", line 1936, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "main.py", line 28, in index
    return jsonify({"foo": uuid4()})
  File "/home/moose/.pyenv/versions/3.8.3/lib/python3.8/site-packages/flask/json/__init__.py", line 370, in jsonify
    dumps(data, indent=indent, separators=separators) + "\n",
  File "/home/moose/.pyenv/versions/3.8.3/lib/python3.8/site-packages/flask/json/__init__.py", line 211, in dumps
    rv = _json.dumps(obj, **kwargs)
  File "/home/moose/.pyenv/versions/3.8.3/lib/python3.8/site-packages/simplejson/__init__.py", line 398, in dumps
    return cls(
  File "main.py", line 14, in encode
    return json.dumps(o)
TypeError: UUID('1f45a2bc-c964-48f0-b2f5-9ef7a2557966') is not JSON serializable

What versions are you using?

• OS: Ubuntu 20.04
• Python: Python 3.8.4
• UltraJSON: ujson==3.2.0

[hugovk]

Please see #374 (comment) and 53f85b1:

Removed generic serialization of objects/iterables

The behavior of ujson has always been to try to serialize all objects in
any way possible. This has been quite a deviation from other json
libraries, including Pythons standard json module, and the source of a
lot of confusion and bugs. Removing this quirk moves ultrajson closer to
the expected behavior.

Instead of trying to coerce serialization ultrajson will now throw a
TypeError: "repr(obj) is not JSON serializable" exception.

[MartinThoma]

Does this mean it is impossible to use ujson for Flask in this context, as ujson does not provide a JSONEncoder class?

[MartinThoma]

Is this maybe the reason why ujson / ultrajson seems not to be in any benchmark of https://www.techempower.com/benchmarks/#section=data-r19&hw=ph&test=json ?

[hugovk]

UltraJSON aims to have a similar API to the stdlib json library, I guess you can't do it with that either?

I've not seen those benchmarks before, you'd have to ask TechEmpower :)

[MartinThoma]

UltraJSON aims to have a similar API to the stdlib json library, I guess you can't do it with that either?

You can. You can also do it with simplejson:

>>> from json import JSONEncoder
>>> from simplejson import JSONEncoder
>>> from ujson import JSONEncoder
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: cannot import name 'JSONEncoder' from 'ujson' (/home/math/.pyenv/versions/3.8.4/lib/python3.8/site-packages/ujson.cpython-38-x86_64-linux-gnu.so)

[MartinThoma]

In those cases I don't overwrite the encode function, because it already uses the default json (or simplejson) module.

[MartinThoma]

Looking at simplejson.encode: Does ujson have something similar to "iterencode"?

[JustDevZero]

The behavior of ujson has always been to try to serialize all objects in
any way possible.

That actually makes no sense, if the behaviour of ujson is trying to serialize all objects in any way, if there's no way at least way to do it should be provided by us... I was moving to ujson because it was serializing datetimes and dates and now all the moving has been suddenly stoped because things like this:

No UUID
No datetime
No date
No time

No custom encoder.

If it aims to provide a similar API this behavour it's doing just the oposite. I think this makes even more confusion to the users.

:(

[dshein-alt]

Didn't check in details how about pure Flask application, but in my project based on Flask-RESTX, which one could use ujson by default, I've come up with a solution to replace custom JSON encoder class with just a function that supplied as default argument to dumps call.

Shortly smth like that works for me:

# how it's done in flask_restx library
try:
    from ujson import dumps
except ImportError:
    from json import dumps

# just call dumps(...) if you need it anywhere in your code
...

# support specific objects serialization here
import datetime
from uuid import UUID

def json_default(obj):
    # convert datetime to ISO string representation
    if isinstance(obj, datetime.datetime):
        return obj.isoformat()
    # convert UUID to string
    if isinstance(obj, UUID):
        return str(obj)
    
    return obj

# just supply your custom object conversion function as a fallback 
# to dumps whether it is from standard `json` library or `ujson`
# for Flask-RESTX it is done through configuration
flask_app.config["RESTX_JSON"] = {"default": json_default}