How to exclude null value from pydantic model used by FAST API

[karan-kumar-by]

I am currently using pydantic model as below. It has 2 optional fields description and tax.

from typing import Optional
from fastapi import Body, FastAPI
from pydantic import BaseModel, Field

app = FastAPI()

class Item(BaseModel):
name: str
description: Optional[str] = Field(None, title="The description of the item", max_length=300)
price: float = Field(..., gt=0, description="The price must be greater than zero")
tax: Optional[float] = None

@app.post("/items/")
async def create_item(item: Item):
return item

Problem - How to exclude optional fields from JSON when not supplied as part of the request.

Request -:
{
"name": "A",
"price": 10.21
}

Fast API produces -:
{
"name": "A",
"description": null,
"price": 10.21,
"tax": null
}

Expected -:
{
"name": "A",
"price": 10.21
}

PYDANTIC provides a way (exclude_none flag) to do that but how to make use of that flag when it is being used in fast API?

[karan-kumar-by]

This what pydantic provides - https://pydantic-docs.helpmanual.io/usage/exporting_models/#modeldict

[gyKa]

Try this code:

@app.post("/items/")
async def create_item(item: Item):
    return item.dict(exclude_none=True)

[plankthom]

Using exclude_none doesn't seem to work when a response_model is in mentioned in the route decorator.

@app.post("/items/", response_model=Item)
async def create_item(item: Item):
    return item.dict(exclude_none=True)

Fast api seems to reprocess the dict with the pydantic model

So i settled for overriding the dict method in the model itself

def Item(BaseModel):
   name: str
   description: Optional[str]
   ...
   def dict(self, *args, **kwargs) -> Dict[str, Any]:
        _ignored = kwargs.pop('exclude_none')
        return super().dict(*args, exclude_none=True, **kwargs)

(an actual solution would put this definition in separate subclass of BaseModel for reuse)

Note: just changing the default value of the. exclude_none keyword argument is not enough: it seems FastAPI always sends exclude_none=False as an argument ...

[squat]

You can accomplish this directly with the decorator by setting the response_model_exclude_none parameter, e.g.:

@app.post("/items/", response_model=Item, response_model_exclude_none=True)
async def create_item(item: Item):
    # do something with item
    return item

This was added last year in #1166 and is available since version 0.54.0

[karan-kumar-by]

@squat I need to exclude None from the request body, not from the response model.

`from typing import Optional
from fastapi import Body, FastAPI
from pydantic import BaseModel, Field
from typing import List

app = FastAPI()

class Item(BaseModel):
name: str
description: Optional[str] = Field(None, title="The description of the item", max_length=300)
price: float = Field(..., gt=0, description="The price must be greater than zero")
tax: Optional[float] = None

@app.post("/items/")
async def create_item(item: List[Item] = Body(...)):
return item

`
Request -:
{
"name": "A",
"price": 10.21
}

Fast API produces -:
{
"name": "A",
"description": null,
"price": 10.21,
"tax": null
}

Expected -:
{
"name": "A",
"price": 10.21
}

Question is - How to send the exclude_none or exclude_default flag to the PYDANTIC model?

[squat]

Your initial question was

How to exclude optional fields from JSON

However, you're saying that you're not concerned with the JSON response from the API handler, correct?

I don't understand what you're trying to accomplish. JSON is only relevant in two places here: the serialized request and the serialized response. You can't control the whether or not a field is present in a request's JSON because that comes from the client. Are you worried about JSON in requests coming from clients generated by your OpenAPI spec? Again, this is something out of FastAPI's control.

Maybe it would be helpful if you share when/where you expect this? e.g.:

• do you want JSON to exclude these fields? As mentioned above you can only enforce the response;
• if it's not about JSON, do you want the Python item object inside the handler to exclude these fields? exclude_none is used when turning a Pydantic object into a dict, e.g. right before the handler returns JSON. item is not a dict; it is a Python object of a type that you defined as having a description and tax field. These fields will always be present on the item object, regardless of whether the request JSON had them. Remember, Optional is a bit of a misnomer; it does not mean that having the field is optional but rather that the value can be None. It is defined as shorthand for a union between None and another type.

[karan-kumar-by]

@squat Thanks for the detailed explanation.

Let me ask this way - How to exclude none before the handler returns JSON?

[AthreyVinay]

@plankthom Your solution works for a get operation. On a put, I get the following:

KeyError: 'exclude_none'

Could you kindly confirm? Thanks

[guybartal]

+1

[daipham3213]

@plankthom Your solution works for a get operation. On a put, I get the following:

KeyError: 'exclude_none'

Could you kindly confirm? Thanks

You could check if the attr exist before remove it

def Item(BaseModel):
   name: str
   description: Optional[str]
   ...
   def dict(self, *args, **kwargs) -> Dict[str, Any]:
         if hasattr(kwargs, "exclude_none"):
            _ignored = kwargs.pop("exclude_none")
        return super().dict(*args, exclude_none=True, **kwargs)

[AthreyVinay]

@plankthom Your solution works for a get operation. On a put, I get the following:

KeyError: 'exclude_none'

Could you kindly confirm? Thanks

You could check if the attr exist before remove it

def Item(BaseModel):
   name: str
   description: Optional[str]
   ...
   def dict(self, *args, **kwargs) -> Dict[str, Any]:
         if hasattr(kwargs, "exclude_none"):
            _ignored = kwargs.pop("exclude_none")
        return super().dict(*args, exclude_none=True, **kwargs)

hi @daipham3213

I tried your solution with the following error:

TypeError: pydantic.main.BaseModel.dict() got multiple values for keyword argument 'exclude_none'

[arrunpras]

if someone still looking for this answer, this worked for me

@app.post("/items/") 
async def create_item(item: Item): 
    return item.dict(exclude_unset=True)

[greatvovan]

@squat's suggestion looks great, but I found this solution by @arrunpras (and derived from that response_model_exclude_unset router parameter) more useful, because you often want to have runtime control for which properties to include, and by leaving some properties unset in the model initializer you can achieve exactly that

[FastestMolasses]

@plankthom Your solution works for a get operation. On a put, I get the following:

KeyError: 'exclude_none'

Could you kindly confirm? Thanks

You could check if the attr exist before remove it

def Item(BaseModel):
   name: str
   description: Optional[str]
   ...
   def dict(self, *args, **kwargs) -> Dict[str, Any]:
         if hasattr(kwargs, "exclude_none"):
            _ignored = kwargs.pop("exclude_none")
        return super().dict(*args, exclude_none=True, **kwargs)

This is a better solution

class Item(BaseModel):
    name: str
    description: Optional[str]
    ...
    def dict(self, *args, **kwargs) -> dict[str, Any]:
        """
            Override the default dict method to exclude None values in the response
        """
        kwargs.pop('exclude_none', None)
        return super().dict(*args, exclude_none=True, **kwargs)

[lucasgadams]

We really need this as a built in feature. I find myself writing this dict override all the time. It should be something we can set either on Field or in the Config

[Ceph97]

For newer Pydantic version this works best:

    def model_dump(self):
        """
        This Overrides the default model dump method to exclude None values
        """
        return super().model_dump(exclude_none=True)

[dh-mshedden]

I can't get this to work the way I expect. Is it supposed to be called by FastAPI when the response is being serialized? I have to call it explicitly, which isn't what I need.

[ivanvmoreno]

Agree with the rest of the commenters – I think this use case would be a perfect fit as a Config member:

class Profile(BaseModel):
    first_name: str
    last_name: str
    email: Optional[str] = None
    
    class Config:
        none = "ignore"