Raise a 500 internal server error when validate a pydantic.HttpUrl with wrong schema

[thomassun]

Describe the bug

got a 500 status with "TypeError: Object of type set is not JSON serializable" if I feed a invalid httpurl as Body

To Reproduce

Steps to reproduce the behavior with a minimum self-contained file.

1. Create a file with:

from fastapi import FastAPI
from pydantic import BaseModel, HttpUrl


class Image(BaseModel):
    url: HttpUrl
    name: str


app = FastAPI()


@app.put("/items/image")
async def update_image(image: Image):
    return image

3. Open the browser and call the endpoint PUT /items/image with body value: {"url":"htp://www.google.com","name":"something"}. here the schema of URL is invalid
4. It returns a 500 Internal Server Error.
5. But I expected it to return 422 with a hint of wrong URL format.

Expected behavior

the type validation should know the wrong URL and give a clear indication

Screenshots

If applicable, add screenshots to help explain your problem.

Environment

• OS: [macOS]
• Python Version 3.7.5
• FastAPI Version 0.44.0
• pydantic Version 1.2
• starlette Version 0.12.9

[dmontagu]

Can you run the server with debug=True (in the FastAPI() call) and share the traceback?

[thomassun]

http put localhost:8000/items/image url="htp://www.google.com" name="df"
---- output with traceback -----
HTTP/1.1 500 Internal Server Error
content-length: 1612
content-type: text/plain; charset=utf-8
date: Tue, 03 Dec 2019 08:10:26 GMT
server: uvicorn

File "/Users/guoqingsun/PycharmProjects/fastapi/venv/lib/python3.7/site-packages/starlette/middleware/errors.py", line 156, in call
await self.app(scope, receive, _send)
File "/Users/guoqingsun/PycharmProjects/fastapi/venv/lib/python3.7/site-packages/starlette/exceptions.py", line 81, in call
response = await handler(request, exc)
File "/Users/guoqingsun/PycharmProjects/fastapi/venv/lib/python3.7/site-packages/fastapi/exception_handlers.py", line 22, in request_validation_exception_handler
status_code=HTTP_422_UNPROCESSABLE_ENTITY, content={"detail": exc.errors()}
File "/Users/guoqingsun/PycharmProjects/fastapi/venv/lib/python3.7/site-packages/starlette/responses.py", line 42, in init
self.body = self.render(content)
File "/Users/guoqingsun/PycharmProjects/fastapi/venv/lib/python3.7/site-packages/starlette/responses.py", line 151, in render
separators=(",", ":"),
File "/usr/local/Cellar/python/3.7.5/Frameworks/Python.framework/Versions/3.7/lib/python3.7/json/init.py", line 238, in dumps
**kw).encode(obj)
File "/usr/local/Cellar/python/3.7.5/Frameworks/Python.framework/Versions/3.7/lib/python3.7/json/encoder.py", line 199, in encode
chunks = self.iterencode(o, _one_shot=True)
File "/usr/local/Cellar/python/3.7.5/Frameworks/Python.framework/Versions/3.7/lib/python3.7/json/encoder.py", line 257, in iterencode
return _iterencode(o, 0)
File "/usr/local/Cellar/python/3.7.5/Frameworks/Python.framework/Versions/3.7/lib/python3.7/json/encoder.py", line 179, in default
raise TypeError(f'Object of type {o.class.name} '

[thomassun]

btw, I got it fixed by changing line 172 in pydanic/errors.py, wrapped allowed_schemes with a list:
before: super().init(allowed_schemes=allowed_schemes)
after: super().init(allowed_schemes=list(allowed_schemes))

but this may not the right place to fix it as pydanic/errors.py is not responsible to grantee the error is in JSON format

[dmontagu]

Ah okay, this is actually fixed on master in FastAPI; the change affects this line:

File "/Users/guoqingsun/PycharmProjects/fastapi/venv/lib/python3.7/site-packages/fastapi/exception_handlers.py", line 22, in request_validation_exception_handler
status_code=HTTP_422_UNPROCESSABLE_ENTITY, content={"detail": exc.errors()}

exc.errors() is now wrapped in a call to jsonable_encoder.

The next release of FastAPI should fix this for you (and you'll be able to drop the modification to pydantic).

[thomassun]

Sounds great, thanks

[tazimmerman]

I'm not sure if this is related, but when I return a list of models, some of which have validation errors, I am getting 500 instead of a 422.

Traceback (most recent call last):
  File "/home/tazimmerman/git/project/env/lib/python3.7/site-packages/uvicorn/protocols/http/httptools_impl.py", line 376, in run_asgi
    result = await app(self.scope, self.receive, self.send)
  File "/home/tazimmerman/git/project/env/lib/python3.7/site-packages/fastapi/applications.py", line 140, in __call__
    await super().__call__(scope, receive, send)
  File "/home/tazimmerman/git/project/env/lib/python3.7/site-packages/starlette/applications.py", line 134, in __call__
    await self.error_middleware(scope, receive, send)
  File "/home/tazimmerman/git/project/env/lib/python3.7/site-packages/starlette/middleware/errors.py", line 178, in __call__
    raise exc from None
  File "/home/tazimmerman/git/project/env/lib/python3.7/site-packages/starlette/middleware/errors.py", line 156, in __call__
    await self.app(scope, receive, _send)
  File "/home/tazimmerman/git/project/env/lib/python3.7/site-packages/starlette/exceptions.py", line 73, in __call__
    raise exc from None
  File "/home/tazimmerman/git/project/env/lib/python3.7/site-packages/starlette/exceptions.py", line 62, in __call__
    await self.app(scope, receive, sender)
  File "/home/tazimmerman/git/project/env/lib/python3.7/site-packages/starlette/routing.py", line 590, in __call__
    await route(scope, receive, send)
  File "/home/tazimmerman/git/project/env/lib/python3.7/site-packages/starlette/routing.py", line 208, in __call__
    await self.app(scope, receive, send)
  File "/home/tazimmerman/git/project/env/lib/python3.7/site-packages/starlette/routing.py", line 41, in app
    response = await func(request)
  File "/home/tazimmerman/git/project/env/lib/python3.7/site-packages/fastapi/routing.py", line 140, in app
    exclude_unset=response_model_exclude_unset,
  File "/home/tazimmerman/git/project/env/lib/python3.7/site-packages/fastapi/routing.py", line 72, in serialize_response
    raise ValidationError(errors, field.type_)
pydantic.error_wrappers.ValidationError: 54 validation errors for AssetModel
[ ... normal pydantic messages follow ... ]

If this doesn't sound related, I'll open a separate issue and put together a reproducible example. I can also try downgrading FastAPI & Pydantic to see if the recent upgrade to 0,.440 and 1.2 (respectively) made a difference.

Environment

OS: [Linux]
Python Version 3.7.3
FastAPI Version 0.44.0
pydantic Version 1.2
starlette Version 0.12.9

[dmontagu]

@tazimmerman I think this is a separate issue -- it would be great if you could produce a small reproducible example in a new issue. (But make sure 0.44.1 doesn't just fix it.)

@thomassun hopefully 0.44.1 fixes your problem; let us know if you still have an issue after upgrading.

[tazimmerman]

I upgraded to 0.44.1 but that didn't fix it. Thanks to the incredibly easy to use API :) I was able to put together a simple example to reproduce the error. I'll open it as a separate issue.

[tiangolo]

Thanks for the discussion here!

@thomassun was your problem solved?

[github-actions[bot]]

Assuming the original issue was solved, it will be automatically closed now. But feel free to add more comments or create new issues.