Pydantic: Detect if a field value is missing or given as null
Question:
I want to allow users to selectively update fields using PUT calls. On the pydantic model, I have made the fields Optional. In the FastAPI handler if the model attribute is None, then the field was not given and I do not update it.
The problem with this approach is that there is no way for the client to "blank out" a field that isn’t required for certain types.
In particular, I have date fields that I want clients to be able to clear by sending in a null in the JSON. How can I detect the difference between the client sending null or the client not sending the field/value at all? The model attribute is just None in either case.
Answers:
The pydantic documentation desccribes two options that can be used with the .dict()
method of models.
-
exclude_unset
: whether fields which were not explicitly set when creating the model should be excluded from the returned dictionary; default False. Prior to v1.0, exclude_unset was known as skip_defaults; use of skip_defaults is now deprecated
-
exclude_defaults
: whether fields which are equal to their default values (whether set or otherwise) should be excluded from the returned dictionary; default False
So you can create a model class with optional fields:
from typing import Optional
from pydantic import BaseModel
class MyModel(BaseModel):
foo: Optional[int] = None
bar: Optional[int] = None
And still generate a dict with fields explicitely set to None, but without default values:
baz = MyModel(foo=None)
assert baz.dict(exclude_unset=True) == {"foo": None}
baz = MyModel(bar=None)
assert baz.dict(exclude_unset=True) == {"bar": None}
You can check obj.__fields_set__
to see whether the value was missing or not.
from typing import Optional
from pydantic import BaseModel
class Foo(BaseModel):
first: Optional[int] = None
second: Optional[int] = None
foo = Foo.parse_raw('{"first": null}')
assert foo.first is None and foo.second is None
assert foo.__fields_set__ == {"first"}
Another way to differentiate between a provided None
and no value is to use a @root_validator(pre=True)
; e.g., to allow nullable non-optional fields.
To be honest I’m not sure if it’s suited for your use case, but I was pointed here by various google searches as it answers your question title, so I’ll post my answer anyway.
The advantage is that it’s checked "inside the model" and can be validated, e.g., via MyModel.validate(my_dict)
without external extra assertions.
from typing import Optional, root_validator
from pydantic import BaseModel
class MyModel(BaseModel):
foo: Optional[int]
bar: Optional[int]
@root_validator(pre=True)
def _check_whether_foo_present(cls, values):
if not "foo" in values:
# do something, e.g.:
raise ValueError('"foo" was not provided to the model')
return values
I want to allow users to selectively update fields using PUT calls. On the pydantic model, I have made the fields Optional. In the FastAPI handler if the model attribute is None, then the field was not given and I do not update it.
The problem with this approach is that there is no way for the client to "blank out" a field that isn’t required for certain types.
In particular, I have date fields that I want clients to be able to clear by sending in a null in the JSON. How can I detect the difference between the client sending null or the client not sending the field/value at all? The model attribute is just None in either case.
The pydantic documentation desccribes two options that can be used with the .dict()
method of models.
-
exclude_unset
: whether fields which were not explicitly set when creating the model should be excluded from the returned dictionary; default False. Prior to v1.0, exclude_unset was known as skip_defaults; use of skip_defaults is now deprecated -
exclude_defaults
: whether fields which are equal to their default values (whether set or otherwise) should be excluded from the returned dictionary; default False
So you can create a model class with optional fields:
from typing import Optional
from pydantic import BaseModel
class MyModel(BaseModel):
foo: Optional[int] = None
bar: Optional[int] = None
And still generate a dict with fields explicitely set to None, but without default values:
baz = MyModel(foo=None)
assert baz.dict(exclude_unset=True) == {"foo": None}
baz = MyModel(bar=None)
assert baz.dict(exclude_unset=True) == {"bar": None}
You can check obj.__fields_set__
to see whether the value was missing or not.
from typing import Optional
from pydantic import BaseModel
class Foo(BaseModel):
first: Optional[int] = None
second: Optional[int] = None
foo = Foo.parse_raw('{"first": null}')
assert foo.first is None and foo.second is None
assert foo.__fields_set__ == {"first"}
Another way to differentiate between a provided None
and no value is to use a @root_validator(pre=True)
; e.g., to allow nullable non-optional fields.
To be honest I’m not sure if it’s suited for your use case, but I was pointed here by various google searches as it answers your question title, so I’ll post my answer anyway.
The advantage is that it’s checked "inside the model" and can be validated, e.g., via MyModel.validate(my_dict)
without external extra assertions.
from typing import Optional, root_validator
from pydantic import BaseModel
class MyModel(BaseModel):
foo: Optional[int]
bar: Optional[int]
@root_validator(pre=True)
def _check_whether_foo_present(cls, values):
if not "foo" in values:
# do something, e.g.:
raise ValueError('"foo" was not provided to the model')
return values