remove ujson dependency (it is dead)

[hnykda]

FastAPI is using ujson which has not been actively developed for about 3 years by now. Here is a relevant issue. It's starting to be problematic to actually compile ujson on newer platforms.

Possible alternative maybe orjson.

[dmontagu]

I would be in favor of recommending orjson instead of ujson.

I recall the creator of orjson reaching about this many months ago and there was some confusion due to slow responses, and also some issues on the pydantic side, but I think many/most of the barriers to usage have since been removed. If orjson seems to be taking over as the go-to high-performance json library, I think it would make sense to use that for development and recommend it.

To the extent that there is any difference in json encoding/decoding between json and orjson (or ujson and orjson), it would probably be best to document it.

I'd be willing to review a PR making this change (including updates to the docs). (I'd leave the ultimate decision about merging to @tiangolo though.)

[tiangolo]

Yeah, I've been intending to do it for quite some time now... 😅

In fact, you can sort of do it right now (and since several versions ago).

---

Here's my plan:

I want to export everything from Starlette, to avoid confusions of the type "do I import this from FastAPI or from Starlette?", I've seen reports of that and even I've struggled with that. For example, FastAPI has its own HTTPException.

With that, I want to have a response ORJSONResponse exported from fastapi.responses, among Starlette's responses.

In fact, I added tests for creating custom responses (as part of a PR), testing specifically creating orjson responses:

fastapi/tests/test_default_response_class.py

Lines 9 to 13 in f803c77

	class ORJSONResponse(JSONResponse):
	media_type = "application/x-orjson"
	def render(self, content: Any) -> bytes:
	return orjson.dumps(content)

And you can replicate creating that response class and using it. It's just 3 lines (you would remove the custom media_type), but still, it should be provided by FastAPI (among all the other responses from Starlette).

app = FastAPI(default_response_class=ORJSONResponse)

---

Also, as a side note, UJSON is not really used by default anywhere. It's optional, and you would have to use the UJSONResponse class for it to be used, the same way you would use the (new/custom/future) ORJSONResponse.

[hnykda]

Well, that's elegant sir 🙇‍♂️ 👏 .

Also, as a side note, UJSON is not really used by default anywhere. It's optional, and you would have to use the UJSONResponse class for it to be used, the same way you would use the (new/custom/future) ORJSONResponse.

Then I don't understand why it's in requirements. It's written that it's an optional dependency for Pydantic, but it seems that Pydantic has already moved away from ujson here. Seems that their requirements nor setup.py don't have it anymore.

[dmontagu]

@hnykda the “all” set of requirements there includes everything necessary to make use of all documented features, including those that are optional. It is intended more for development/reference purposes than for use by consumers.

[hnykda]

Ah, got it! Thanks. I believe this can be closed then.

Personally, I would not reference any obsolete dependencies in my project even if they are optional (especially "future-looking" like Python 3.6+, latest libs, ...) . Reasons for that: helping the community to get rid dangerous/old/unmaintained libs and to avoid possibly repealing people who do not like to install stuff which are referencing such unmaintained projects.

[dagnelies]

Apparently, ujson found a new maintainer and is now alive and kicking again. Well, dunno about the kicking part, but releases are pretty fresh:

https://pypi.org/project/ujson/

and repo seems active again. Just mentioning this. Dunno which one is the better lib.

[dagnelies]

Nevermind, apparently orjson is a fine choice too. Apparently one of the fastest.

[ahmadazizi]

Just to mention, installing fastapi on windows platform (python 3.9) fails, since orjson neither provide the compatible wheel nor could be compiled.

[jacopofar]

On Debian stable installing orjson currently fails because it targets manylinux2014 which is supported by pip since version 19.3, but Debian stable install pip 18.1 (if installed through Debian repos python3-pip).

In this case running python3 -m pip install --upgrade pip works, but the error can be tricky to detect

[ahtik]

@ahmadazizi, orjson wheels are now available also for Windows Python 3.9.

[ycd]

Since Starlette dropped support for UJSONResponse in encode/starlette#1047 (Requires Starlette 0.14.1), we can remove it too, ujson causes build fails on many distributions due C Dependencies (See #2811 ).

This require changes in #2335, since it re-adds UJSONResponse.

[omidmaldar]

Regarding ujson, there is a security alert reported with moderate severity which is patched in 5.1.0. Fastapi 0.74.0 requires ujson (>=4.0.1,<5.0.0) which is not safe.

[melt7777]

Hello all, I did find a workaround which allows us to build images without the ujson. I am using the python-poetry package manager, instead of requirements.txt.

FastAPI is fantastic so far, thank you all for your hard work on this project.

Here, roughly the steps I used:

1. poetry init
2. poetry add fastapi -E all - it installed the ujson package and placed the ujson bits into the pyproject.toml and the newly created poetry.lock file.
3. Manually edit the .toml and .lock files and remove all the ujson items.
4. Build image as usual and it will not have the ujson or its vuln in the image.

Hope this helps someone until they scan make the permanent fix.

Thank you,

~ melt

[alexted]

Are you sure the ujson is EOL? According to its repository, the last release was in mid-September, and the last commit was 3 weeks ago - the project seems to be quite alive.
Another point is that it is inferior to orjson in performance. In that case, it can be left as an option.

[hofrob]

Just as an update. This is what the pypi page says:

[!WARNING] UltraJSON's architecture is fundamentally ill-suited to making changes without risk of introducing new security vulnerabilities. As a result, this library has been put into a maintenance-only mode. Support for new Python versions will be added and critical bugs and security issues will still be fixed but all other changes will be rejected. Users are encouraged to migrate to orjson which is both much faster and less likely to introduce a surprise buffer overflow vulnerability in the future.

Additionally, I just want to mention that in my project, ujson was added as a dependency after upgrading fastapi to 0.111.0. Is that expected? Does this have something to do with the refactoring for fastapi-slim and fastapi-slim[standard]?

[hofrob]

Does this have something to do with the refactoring for fastapi-slim and fastapi-slim[standard]?

Yes, it does. I had to replace fastapi as a dependency with fastapi-slim.