How to filter out NaN by pydantic

Question

How to filter out NaN in pytdantic float validation?

from pydantic import BaseModel

class MySchema(BaseModel):
    float_value: float

Asked By: Ryabchenko Alexander

||

Source

Answer 1

import math
from pydantic import BaseModel, validator

class MySchema(BaseModel):
    float_value: float

    @validator('*', pre=True)
    def split_str(cls, v):
        if isinstance(v, float):
            if math.isnan(v):
                raise ValueError("value can't be Not-a-Number (NaN)")
            return v
        return v

Answered By: Ryabchenko Alexander

Answer 2

Define your custom type for validations, is well documented at pydantic:

class NoNanFloat(float):
    
    @classmethod
    def __get_validators__(cls):
        yield cls.validate
        
    @classmethod
    def __modify_schema__(cls, field_schema):
        # you can ommit this method
        field_schema.update(
            examples=['24.2,15.2'],
        )

    @classmethod
    def validate(cls, v):
        if not isinstance(v, float):
            raise TypeError('float required')
        if v!=v: # you can use here also maths.isnan(v):
            raise ValueError("value can't be Not-a-Number (NaN)")
        return cls(v)

    def __repr__(self):
        # you can also ommit this method, but it looks good when printing. 
        return f'NoNanFloat({super().__repr__()})'
    
class MySchema(BaseModel):
    no_nan_float_value: NoNanFloat
    other_float_value: float
    other: Any

This approach has many advantages, as it allows you to have two types of "floats" depending on your needs, so you can have some allowing nan and others that doesn’t.

I also allows you to have the "Any" type accepting nans, and unions of types behaving as expected.

Answered By: Ziur Olpa

Answer 3

You can use confloat and set either the higher limit to infinity or the lower limit to minus infinity. As all numeric comparisons with NaN return False, that will make pydantic reject NaN, while leaving all other behaviour identical (including parsing, conversion from int to float, …).

from pydantic import BaseModel, confloat

class MySchema(BaseModel):
    float_value: confloat(ge=-float('inf'))
    # or:
    # float_value: confloat(le=float('inf'))

Note: you could additionally exclude infinity values by using the gt and lt arguments of confloat instead of ge and le.

Testing:

m = MySchema(float_value=float('nan'))

Output:

pydantic.error_wrappers.ValidationError: 1 validation error for MySchema
float_value
  ensure this value is greater than or equal to -inf (type=value_error.number.not_ge; limit_value=-inf)

Answered By: ye olde noobe

Answer 4

The up to date answer for pydantic versions >= 1.10 is to use confloat and set its allow_inf_nan accordingly:

class Finite(BaseModel):
    value: confloat(allow_inf_nan=False)

With that, the arguments float("inf"), -float("inf"), float("nan"), and -float("inf") will result in an ValidationError.

See the documentation of confloat.

Answered By: MaxPowers

How to filter out NaN by pydantic

Question:

Answers: