In Pydantic, how do I set env_ignore_empty to True

Question:

I’m currently trying to use Pydantic to check if JSON received is in the right format, and it’s one of those where we have 30, 40 fields coming in the JSON, and they have decided that if an amount (for example) is empty, they will provide a null string. Of course, when I check for a float, even if it is Optional, the system finds a string.

So if I have

from typing import Optional
from pydantic import BaseModel


class Balance(BaseModel):
    outstanding: float
    paidtodate: Optional[float] = None


data1 = Balance(**{"outstanding": 1000.00})
data2 = Balance(**{"outstanding": 1000.00, "paidtodate": 500.00})
data3 = Balance(**{"outstanding": 1000.00, "paidtodate": ""})

Problem is that data3 fails, because (rigthly so) paidtodate is a string, but I want to be able to say, if it’s empty string, just accept it as none.

There seems to be an environment variable called env_ignore_empty, but I can’t for the life of me figure out how to set it.

Of course, if there is some other way, would love that as well.

Using pydantic_core-2.16.3-cp39-cp39-macosx_11_0_arm64.whl

Asked By: vrghost

||

Answers:

All you need is to convert empty strings to None and let the pydantic do the rest, i.e. validate it against Optional[float]:

from typing import Optional
from pydantic import BaseModel, field_validator

class Balance(BaseModel):
    outstanding: float
    paidtodate: Optional[float] = None

    # add more field names after 'paidtodate' or use '*' for all fields
    @field_validator('paidtodate',  mode='before')
    @classmethod
    def _emtpy_to_none(cls, field_value):
        if field_value == "":
            return None
        return field_value   # unchanged

Docs: https://docs.pydantic.dev/latest/concepts/validators/#field-validators


The updated version below should enable the preprocessing for all fields defined as Optional. Experimental code warning, I’m not a pydantic guru.

from typing import Optional, get_origin, get_args, Union
from pydantic import BaseModel, field_validator


class Balance(BaseModel):
    outstanding: float
    paidtodate: Optional[float] = None

    @field_validator('*', mode='before')
    @classmethod
    def _emtpy_to_none(cls, field_value, vinfo):
        if field_value == "":
            # check for "Optional"
            fa = cls.model_fields[vinfo.field_name].annotation
            # credit: https://stackoverflow.com/a/58841311/5378816
            if get_origin(fa) is Union and type(None) in get_args(fa)
                return None
        return field_value
Answered By: VPfB

You can do something like the following to avoid a "custom" validator:

from pydantic_settings import BaseSettings


class Settings(BaseSettings):
    model_config = {"env_ignore_empty": True}
    SOME_VALUE: float | None = None

Unfortunately, this seem to only be supported for BaseSettings and not BaseModel atm.

Answered By: Janus Heide
Categories: questions Tags: ,
Answers are sorted by their score. The answer accepted by the question owner as the best is marked with
at the top-right corner.