Hypothesis's pandas extension is slow compared to pydantic

[tmcclintock]

This is a head-to-head comparison between pydantic and the pandas extension in hypothesis, where we are generating a single-row dataframe with hypothesis (since pydantic only generates one model instance at a time).

"""Test hypothesis with pydantic basemodel."""

from hypothesis import given, strategies
from hypothesis.extra import pandas as hyp_pandas

from pydantic import BaseModel


class PydanticExample(BaseModel):

    feature1: int
    feature2: float
    feature3: str



df_strat = hyp_pandas.data_frames(
    columns=[
        hyp_pandas.column("feature1", dtype=int),
        hyp_pandas.column("feature2", dtype=float),
        hyp_pandas.column("feature3", dtype=str),
    ],
    index=hyp_pandas.range_indexes(min_size=1, max_size=1)
)


@given(strategies.builds(PydanticExample))
def test_pydantic_example(example):
    PydanticExample.validate(example)


@given(df_strat)
def test_hypothesis_pandas_example(example):
    assert "feature1" in example
    assert len(example) == 1

I ran it with: pytest test_speed.py --durations=0 -v

Takeaway: hypothesis is 9-10x slower than pydantic as seen by this output:

============================================================ slowest durations =============================================================
0.78s call     test_speed.py::test_hypothesis_pandas_example
0.08s call     test_speed.py::test_pydantic_example

For background, this issue was originally brought up in this issue in the pandera project. pandera (basically) has a thin wrapper around the pandas.data_frames() method from hypothesis.

My questions:

1. Do maintainers have a theory on what the cause of this speed discrepancy might be?
2. Is there appetite for optimizing this logic to make it faster? I'm happy to help where I can btw.

Thank you!

[Zac-HD]

1. There's just way way more logic involved in generating a dataframe, due to the possibility of having many rows, interacting constraints, etc. By contrast the pydantic model case is three built-in types, one function call, and no dtype conversions or validation or anything.
2. Yes, I'd be absolutely delighted to accept PRs for performance improvements - only caveat is if it makes future maintenance substantially harder.

[Zac-HD]

See also unionai-oss/pandera#404

[Zac-HD]

Closing this because there's no planned action on our end.