Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

misc: determine a method to globally handle custom json serialization using httpx #586

Open
cmyui opened this issue Feb 13, 2024 · 2 comments
Assignees
Labels
bug Something isn't working code quality Something is implemented poorly

Comments

@cmyui
Copy link
Member

cmyui commented Feb 13, 2024

At the moment, our server has a couple of locations where objects of types such as datetime are not able to be serialized into JSON due to a lack of support in the stdlib json module. httpx -- the http client we use for outgoing requests, relies specifically on the stdlib json module.

I see a couple of ways we can tackle this:

  1. Use post(headers={'Content-Type': 'application/json'}, data=custom_json_dumps(...)) in each call. Never use post(json=...). Seems quite error prone/duplicative.
  2. Subclass AsyncClient, override the build_request method, copy most from httpx but use a custom Request which overrides __init__ to perform option 1's suggestion automatically before yielding to super().__init__.
  3. Monkey-patch httpx._content.encode_json to provide a custom JSONEncoder subclass to json.dumps(cls=...).
  4. Implement the httpx maintainer's proposal to httpx to implement support for AsyncClient(request_class=..., response_class=...). Do the same __init__ override as suggested in option 1.

My preference is probably option 4 -- seems the best for the overall ecosystem. We could potentially do one of the other options in the short term as I believe this is currently having an effect on production.

@cmyui cmyui added the triage This issue or pull request needs sorting. label Feb 13, 2024
@cmyui cmyui self-assigned this Feb 13, 2024
@cmyui
Copy link
Member Author

cmyui commented Feb 13, 2024

cc: @tsunyoku @NiceAesth @minisbett

@NiceAesth NiceAesth added bug Something isn't working code quality Something is implemented poorly and removed triage This issue or pull request needs sorting. labels Feb 13, 2024
@minisbett
Copy link
Contributor

minisbett commented Feb 26, 2024

potential hotfix:

import orjson
import httpx._content

from typing import Any, Tuple, Dict

def encode_json(json: Any) -> Tuple[Dict[str, str], httpx._content.ByteStream]:
    body = orjson.dumps(json)
    content_length = str(len(body))
    content_type = "application/json"
    headers = {"Content-Length": content_length, "Content-Type": content_type}
    return headers, httpx._content.ByteStream(body)

httpx._content.encode_json = encode_json

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working code quality Something is implemented poorly
Projects
None yet
Development

No branches or pull requests

3 participants