Bump pydantic and pyarrow version

[Atharva-Phatak]

I was trying to install evidently with the latest versions of pydantic (v2) and pyarrow. I found we are still using version 11dev of pyarrow and currently pyarrow is on version 14. I think we can bump up both the dependencies and see massive perfomance gains.

[rxm7706]

@Atharva-Phatak
came here to say the same thing - but focused on pyarrow - because we ran into https://nvd.nist.gov/vuln/detail/CVE-2023-47248 - and while trying to upgrade to pyarrow 14.0.1 - evidently was a blocker.

Also in case this helps others - since the CVE affects all PyArrow versions from 0.14.0 to 14.0.0
https://github.com/pitrou/pyarrow-hotfix provides a good work around for both pip and conda users to keep greater pyarrow support (not needing to pin to only pyarrow >14.0.1)

pip install pyarrow_hotfix
conda install -c conda-forge pyarrow-hotfix

It probably makes sense to release a pyarrow bump and version upgrade separately as Pyarrow upgrade will probably give more returns in terms of performance - and address the open security CVE.

On Pydantic - Just one note

pydantic and fastapi are interdependent - with some complexities
See https://github.com/conda-forge/fastapi-feedstock/blob/main/recipe/meta.yaml#L26C1-L26C68
- pydantic >=1.7.4,!=1.8,!=1.8.1,!=2.0.0,!=2.0.1,!=2.1.0,<3.0.0

1. supporting only pydantic >2 will affect dependencies and break integrations that are still pinned to pydantic <2
2. supporting both will be ideal - but was a bit extra work (based on our experience)

thank you and regards

[blueoceancruiser]

Not having Pydantic v2 seems to impact a lot of packages including fastapi, AutoGluon and anything using lightning, space, thinc or weasel. Any idea when this might be addressed? I wish I could help, but that's way above my pay grade. ;-)

[mike0sv]

We removed pyarrow dependency in #885 and replaced it with pandas[parquet] - which means you can use any pyarrow version pandas is comfortable with.
Regarding pydantic, migration to v2 is not in our current plans. However, we did #792 so that you can use evidently with pydantic v2 (evidently models will still use pydantic.v1 classes)

[dylangreenleaf]

Hoping to get some guidance on how I can use evidently with pydantic v2. The install_requires block within setup.py specifies "pydantic<2" which seems problematic, but perhaps I am missing something. When trying to add evidently as a dependency to my existing poetry project I get the following error:

➜  poetry add evidently  
Using version ^0.4.13 for evidently

Updating dependencies
Resolving dependencies... (0.1s)

Because no versions of evidently match >0.4.13,<0.5.0
 and evidently (0.4.13) depends on pydantic (<2), evidently (>=0.4.13,<0.5.0) requires pydantic (<2).
So, because [my-package] depends on both pydantic (^2.1.1) and evidently (^0.4.13), version solving failed.

[maciejmajerczyk]

I added PR #942 with fixing of that issue. It's kinda blocker for us as we use pydantic>2 across projects

[emeli-dral]

Hi there! As evidently ui service depends of FastAPI and FastAPI does not work with pydantic 2 installed tiangolo/fastapi#10360 we cannot bump pydantic version right now.

However we see that pydantic 2 is very much needed. This is why we now replace FastAPI with litestar. After this replacement we will be able to bump pydantic version, as litestar has no issues with that. I believe it will be finished during the next couple of weeks.

However, if you use evidently to generate Reports or TestSuites without running evidently ui or if you calculate Reports or TestSuites in the one environment but run evidently ui service in the other one, you can install pydantic 2 after you installed evidently in the environment where you do not need to run evidently ui. This is something you can do now before we released evidently on litestar.

I'm sorry it is taking us so long to fix this.

[eduardo-lourenco]

Hi @emeli-dral, I am also running into this issue. Wondering if the timeline to get it fixed is still the same or if there's any developments? Thanks for the hard work!

[emeli-dral]

Hi @eduardo-lourenco,
This is a part of the next release!
We will do our best to make publish it this week 🙏

[JoaquimEsteves]

Hey, just ran into this issue.

FastAPI shouldn't have any issues with pydantic-v2 - according this issue. Note that that issue is older than the one being discussed here.

At $WORK we're happily using both FastAPI with pydantic-v2.

So I don't see the need for you folks to replace FastAPI with something else at all.

In your requirements.dev you seem to have some sort of limitation on the FastAPI version - I'd recommend changing that.

It also feels bizarre for a dev dependency to not be a dev dependency at all - as it's stopping people from using your tool.

If PRs are welcome I'd be happy to contribute - some poor soul will have to spend a few hours tweaking the dependencies but this shouldn't be too hard.

[emeli-dral]

Hi @JoaquimEsteves ,

The issue is a bit more nuanced than just compatibility between Pydantic v2 and FastAPI - it is the inability to enforce a specific Pydantic model version when using FastAPI with Pydantic v2.

To sum up:

• Evidently uses Pydantic model version 1 for validations inside the library. (The complete migration would not be easily feasible - and probably not yet needed).
• There have been major changes between Pydantic v1 and v2, but Pydantic v2 supports both model types. So, you can install Pydantic v2 and use the previous model version (v1) inside it without breaking changes.
• Because of this, Evidently itself can work with both Pydantic v1 and Pydantic v2 (using the v1 model type shipped within the v2 version). Supporting both versions is also great for our users since many remain on Pydantic v1 in their environment.
• However, with FastAPI, there is no option to support working with Pydantic model v1 while having v2 installed. This is a known issue, but it has not been solved on the FastAPI side - which is fair, since we all have a lot of features to build :)

As a result, we decided to replace the FastAPI dependency for litestar. On top of this, we experienced some other issues with FastAPI as we further develop the UI and collector service (where it is used). So we expect additional performance improvements by migrating to Litestar - solving two issues at once.

You can see that this PR is already in the final stages: #995. It was just merged and I expect to release it very soon. Hopefully today🤞

As mentioned, in the meantime, if you only work with Evidently Reports / Test Suites (without self-hosting the UI), you can use a workaround by first installing Evidently and then installing Pydantic 2 after it in the same environment.

And if you see something worth contributing on - feel free to send the PR / open a separate issue, you are very welcome to contribute!

[emeli-dral]

Hey @JoaquimEsteves , @dylangreenleaf , @blueoceancruiser , @maciejmajerczyk , @eduardo-lourenco !
We just released a new version 0.4.16 where you can have pydantic 2 😅

I'm sorry it took us so long! Please feel free to give it a try and let us know if you encounter any more problems with pydantic (or anything else)😊