Can I override fields from a Pydantic parent model to make them optional?
Question:
I have two pydantic classes like this.
class Parent(BaseModel):
id: int
name: str
email: str
class ParentUpdate(BaseModel):
id: Optional[int]
name: Optional[str]
email: Optional[str]
Both of these are practically the same but the Parent class makes all fields required.
I want to use the Parent class for POST request body in FastAPI, hence all fields should be required. But I want to use the latter for PUT request body since the user can set selective fields and the remaining stays the same.
I have taken a look at Required Optional Fields but they do not correspond to what I want to do.
If there was a way I could inherit the Parent class in ParentUpdate and modified all the fields in Parent to make them Optional that would reduce the clutter. Additionally, there are some validators present in the Parent class which I have to rewrite in the ParentUpdate class which I also want to avoid.
Is there any way of doing this?
Answers:
You can make optional fields required in subclasses, but you cannot make required fields optional in subclasses. In fastapi author tiangolo’s boilerplate projects, he utilizes a pattern like this for your example:
class ParentBase(BaseModel):
"""Shared properties."""
name: str
email: str
class ParentCreate(ParentBase):
"""Properties to receive on item creation."""
# dont need id here if your db autocreates it
pass
class ParentUpdate(ParentBase):
"""Properties to receive on item update."""
# dont need id as you are likely PUTing to /parents/{id}
# other fields should not be optional in a PUT
# maybe what you are wanting is a PATCH schema?
pass
class ParentInDBBase(ParentBase):
"""Properties shared by models stored in DB - !exposed in create/update."""
# primary key exists in db, but not in base/create/update
id: int
class Parent(ParentInDBBase):
"""Properties to return to client."""
# optionally include things like relationships returned to consumer
# related_things: List[Thing]
pass
class ParentInDB(ParentInDBBase):
"""Additional properties stored in DB."""
# could be secure things like passwords?
pass
Yes, I agree this is incredibly verbose and I wish it wasn’t. You still likely end up with other schemas more specific to particular forms in your UI. Obviously, you can remove some of these as they aren’t necessary in this example, but depending on other fields in your DB, they may be needed, or you may need to set defaults, validation, etc.
In my experience for validators, you have to re-declare them but you can use a shared function, ie:
def clean_article_url(cls, v):
return clean_context_url(v.strip())
class MyModel(BaseModel):
article_url: str
_clean_url = pydantic.validator("article_url", allow_reuse=True)(clean_article_url)
I apologize in advance, and I’m definitely sure that is a horrible workaround, but it worked for me:
def make_child_fields_optional(parent_class: Type[BaseModel], child_class: Type[BaseModel]):
for key in parent_class.__fields__:
child_class.__fields__.get(key).required = False
class BasePerson(BaseModel):
name: str
email: str
login: str
class UpdatePerson(BasePerson):
pass # or whatever
make_child_fields_optional(BasePerson, UpdatePerson)
My advice is to not invent difficult schemas, I was also interested in pydantic capabilities, but all of them look very ugly and hard to understand (or even not intended for some tasks and have constraints).
See Python pydantic, make every field of ancestor are Optional
Answer from pydantic maintainer
As already outlined in an answer to a similar question, I am using the following approach (credit goes to Aron Podrigal):
import inspect
from pydantic import BaseModel
def optional(*fields):
"""Decorator function used to modify a pydantic model's fields to all be optional.
Alternatively, you can also pass the field names that should be made optional as arguments
to the decorator.
Taken from https://github.com/samuelcolvin/pydantic/issues/1223#issuecomment-775363074
"""
def dec(_cls):
for field in fields:
_cls.__fields__[field].required = False
return _cls
if fields and inspect.isclass(fields[0]) and issubclass(fields[0], BaseModel):
cls = fields[0]
fields = cls.__fields__
return dec(cls)
return dec
In your example you’d use it like this:
@optional
class ParentUpdate(Parent):
pass
Overriding fields is possible and easy. (Somebody mentioned it is not possible to override required fields to optional, but I do not agree).
This example works without any problems:
class Parent(BaseModel):
id: int
name: str
email: str
class ParentUpdate(Parent): ## Note that this inherits 'Parent' class (not BaseModel)
id: Optional[int] # this will convert id from required to optional
For my case creating a new class was the only solution that worked, but packed into a function it is quite convenient:
from pydantic import BaseModel, create_model
from typing import Optional
def make_optional(baseclass):
# Extracts the fields and validators from the baseclass and make fields optional
fields = baseclass.__fields__
validators = {'__validators__': baseclass.__validators__}
optional_fields = {key: (Optional[item.type_], None) for key, item in fields.items()}
return create_model(f'{baseclass.__name__}Optional', **optional_fields, __validators__=validators)
class Parent(BaseModel):
id: int
name: str
email: str
ParentUpdate = make_optional(Parent)
Comparing after and before:
Parent.__fields__
{'id': ModelField(name='id', type=int, required=True),
'name': ModelField(name='name', type=str, required=True),
'email': ModelField(name='email', type=str, required=True)}
ParentUpdate.__fields__
{'id': ModelField(name='id', type=Optional[int], required=False, default=None),
'name': ModelField(name='name', type=Optional[str], required=False, default=None),
'email': ModelField(name='email', type=Optional[str], required=False, default=None)}
It does work, and also it allows you to filter out some fields of the class if it is required.
Moreover for FastApi you can directly use make_optional(Parent) as the type-hint in the API call and that will generate the documentation correctly. Another advantage of this approach is that it can reduce the boilerplate a lot.
I have two pydantic classes like this.
class Parent(BaseModel):
id: int
name: str
email: str
class ParentUpdate(BaseModel):
id: Optional[int]
name: Optional[str]
email: Optional[str]
Both of these are practically the same but the Parent class makes all fields required.
I want to use the Parent class for POST request body in FastAPI, hence all fields should be required. But I want to use the latter for PUT request body since the user can set selective fields and the remaining stays the same.
I have taken a look at Required Optional Fields but they do not correspond to what I want to do.
If there was a way I could inherit the Parent class in ParentUpdate and modified all the fields in Parent to make them Optional that would reduce the clutter. Additionally, there are some validators present in the Parent class which I have to rewrite in the ParentUpdate class which I also want to avoid.
Is there any way of doing this?
You can make optional fields required in subclasses, but you cannot make required fields optional in subclasses. In fastapi author tiangolo’s boilerplate projects, he utilizes a pattern like this for your example:
class ParentBase(BaseModel):
"""Shared properties."""
name: str
email: str
class ParentCreate(ParentBase):
"""Properties to receive on item creation."""
# dont need id here if your db autocreates it
pass
class ParentUpdate(ParentBase):
"""Properties to receive on item update."""
# dont need id as you are likely PUTing to /parents/{id}
# other fields should not be optional in a PUT
# maybe what you are wanting is a PATCH schema?
pass
class ParentInDBBase(ParentBase):
"""Properties shared by models stored in DB - !exposed in create/update."""
# primary key exists in db, but not in base/create/update
id: int
class Parent(ParentInDBBase):
"""Properties to return to client."""
# optionally include things like relationships returned to consumer
# related_things: List[Thing]
pass
class ParentInDB(ParentInDBBase):
"""Additional properties stored in DB."""
# could be secure things like passwords?
pass
Yes, I agree this is incredibly verbose and I wish it wasn’t. You still likely end up with other schemas more specific to particular forms in your UI. Obviously, you can remove some of these as they aren’t necessary in this example, but depending on other fields in your DB, they may be needed, or you may need to set defaults, validation, etc.
In my experience for validators, you have to re-declare them but you can use a shared function, ie:
def clean_article_url(cls, v):
return clean_context_url(v.strip())
class MyModel(BaseModel):
article_url: str
_clean_url = pydantic.validator("article_url", allow_reuse=True)(clean_article_url)
I apologize in advance, and I’m definitely sure that is a horrible workaround, but it worked for me:
def make_child_fields_optional(parent_class: Type[BaseModel], child_class: Type[BaseModel]):
for key in parent_class.__fields__:
child_class.__fields__.get(key).required = False
class BasePerson(BaseModel):
name: str
email: str
login: str
class UpdatePerson(BasePerson):
pass # or whatever
make_child_fields_optional(BasePerson, UpdatePerson)
My advice is to not invent difficult schemas, I was also interested in pydantic capabilities, but all of them look very ugly and hard to understand (or even not intended for some tasks and have constraints).
See Python pydantic, make every field of ancestor are Optional
Answer from pydantic maintainer
As already outlined in an answer to a similar question, I am using the following approach (credit goes to Aron Podrigal):
import inspect
from pydantic import BaseModel
def optional(*fields):
"""Decorator function used to modify a pydantic model's fields to all be optional.
Alternatively, you can also pass the field names that should be made optional as arguments
to the decorator.
Taken from https://github.com/samuelcolvin/pydantic/issues/1223#issuecomment-775363074
"""
def dec(_cls):
for field in fields:
_cls.__fields__[field].required = False
return _cls
if fields and inspect.isclass(fields[0]) and issubclass(fields[0], BaseModel):
cls = fields[0]
fields = cls.__fields__
return dec(cls)
return dec
In your example you’d use it like this:
@optional
class ParentUpdate(Parent):
pass
Overriding fields is possible and easy. (Somebody mentioned it is not possible to override required fields to optional, but I do not agree).
This example works without any problems:
class Parent(BaseModel):
id: int
name: str
email: str
class ParentUpdate(Parent): ## Note that this inherits 'Parent' class (not BaseModel)
id: Optional[int] # this will convert id from required to optional
For my case creating a new class was the only solution that worked, but packed into a function it is quite convenient:
from pydantic import BaseModel, create_model
from typing import Optional
def make_optional(baseclass):
# Extracts the fields and validators from the baseclass and make fields optional
fields = baseclass.__fields__
validators = {'__validators__': baseclass.__validators__}
optional_fields = {key: (Optional[item.type_], None) for key, item in fields.items()}
return create_model(f'{baseclass.__name__}Optional', **optional_fields, __validators__=validators)
class Parent(BaseModel):
id: int
name: str
email: str
ParentUpdate = make_optional(Parent)
Comparing after and before:
Parent.__fields__
{'id': ModelField(name='id', type=int, required=True),
'name': ModelField(name='name', type=str, required=True),
'email': ModelField(name='email', type=str, required=True)}
ParentUpdate.__fields__
{'id': ModelField(name='id', type=Optional[int], required=False, default=None),
'name': ModelField(name='name', type=Optional[str], required=False, default=None),
'email': ModelField(name='email', type=Optional[str], required=False, default=None)}
It does work, and also it allows you to filter out some fields of the class if it is required.
Moreover for FastApi you can directly use make_optional(Parent) as the type-hint in the API call and that will generate the documentation correctly. Another advantage of this approach is that it can reduce the boilerplate a lot.