pydantic 1.8 release #2322

sofroniewn · 2021-02-26T18:25:13Z

🐛 Bug

Looks like the pydantic 1.8 release has broken something. Not sure what yet. Could make a quick pin < 1.8 or quick fix cc @tlambert03

sofroniewn · 2021-02-26T18:28:31Z

Traceback (most recent call last):
  File "examples/add_image.py", line 6, in <module>
    import napari
  File "/Users/nsofroniew/GitHub/napari/napari/__init__.py", line 7, in <module>
    from .viewer import Viewer  # isort:skip
  File "/Users/nsofroniew/GitHub/napari/napari/viewer.py", line 3, in <module>
    from .components import ViewerModel
  File "/Users/nsofroniew/GitHub/napari/napari/components/__init__.py", line 18, in <module>
    from .dims import Dims
  File "/Users/nsofroniew/GitHub/napari/napari/components/dims.py", line 10, in <module>
    class Dims(EventedModel):
  File "/Users/nsofroniew/GitHub/napari/napari/utils/events/evented_model.py", line 78, in __new__
    cls = super().__new__(mcs, name, bases, namespace, **kwargs)
  File "pydantic/main.py", line 287, in pydantic.main.ModelMetaclass.__new__
  File "pydantic/fields.py", line 392, in pydantic.fields.ModelField.infer
  File "pydantic/fields.py", line 327, in pydantic.fields.ModelField.__init__
  File "pydantic/fields.py", line 438, in pydantic.fields.ModelField.prepare
  File "pydantic/fields.py", line 651, in pydantic.fields.ModelField.populate_validators
  File "pydantic/validators.py", line 710, in find_validators
RuntimeError: error checking inheritance of typing.Tuple[float, float, float] (type: Tuple[float, float, float])

Czaki · 2021-02-27T01:17:01Z

There is plan for some fast release (0.4.5.post1 differed from 0.4.5 with one commit) with such pin? It touch also napari in 0.4.5.

sofroniewn · 2021-02-27T04:47:43Z

I think we could start release process for 0.4.6, it's been two weeks. Earliest that would be out is monday/ tuesday though. Thoughts @jni? What do you think we should do?

Czaki · 2021-02-27T09:30:24Z

But it is still few days when the new installation of napari will be broken if the user does not have pydantic already installed.

adamltyson · 2021-03-01T12:07:54Z

I'd vote for releasing a new version of napari to patch this. It took me a while to work out why napari wasn't working in a fresh environment.

VolkerH · 2021-03-01T12:25:17Z

It took me a while to work out why napari wasn't working in a fresh environment.

Same here, just now. As I was building napari inside a Docker container (something I am not that familiar with) I was debugging my Dockerfile for a while before looking for the issue here.
So +1

tlambert03 · 2021-03-01T16:15:49Z

this has been fixed upstream, and will presumably be in the next pydantic release: pydantic/pydantic#2421

(not that we shouldn't release with pydantic<1.8.0)

Czaki · 2021-03-01T16:42:47Z

When new version of pydantic will be released?

tlambert03 · 2021-03-01T16:43:35Z

When new version of pydantic will be released?

at this point, we don't know

sofroniewn · 2021-03-01T16:44:45Z

We've started the 0.4.6 release process now with a pin pydantic<1.8.0, will have an rc to test soon.

sofroniewn · 2021-03-01T16:53:20Z

Ok 0.4.6rc0 has been released and can be tested with pip install --pre napari. Please let us know how that's working on the Zulip release discussion - https://napari.zulipchat.com/#narrow/stream/215289-release/topic/0.2E4.2E6/near/228264146

Carreau · 2021-03-01T23:00:18Z

We've started the 0.4.6 release process now with a pin pydantic<1.8.0, will have an rc to test soon.

That won't totally work, pip has a dependency resolver, and might decide to downgrade napari for end users instead of downgrading pydantic.

Carreau · 2021-03-01T23:04:51Z

That won't totally work, pip has a dependency resolver, and might decide to downgrade napari for end users instead of downgrading pydantic.

Not saying that you should not make a release pinning pydantic to <1.8.0; but that users may end up upgrading pydantic and downgrading napari when manipulating other dependencies, updating plugins.... (had the same issue with IPython/Jedi recently.

tlambert03 · 2021-03-01T23:15:46Z

Interesting... is there any better alternative? Or just something were kinda stuck with?

jni · 2021-03-01T23:35:19Z

I would suggest we release with pydantic!=1.8.0 so that we automatically work with subsequent versions, since the fix is in pydantic master?

Carreau · 2021-03-01T23:55:08Z

Interesting... is there any better alternative? Or just something we're kinda stuck with?

Yeah, it's something we are stuck with. It's an inherent problem of package manager that don't allow to update package meta informations after the release, it also affects conda, but in a lesser form, as you can hotpatch repo index in conda/conda-forge, and the fact that we can't know the future (yet).

It's worse in pip/pypi as the info is in the wheel, and pip recently changed to a resolver.

Project that are 1/2 apps 1/2 libraries (like IPython and napari) also have a tough time as the application side push us toward pinning, and the library side toward letting dependencies version float.

See recent twitter thread, among others, which is about wether one should pin to python<4 or not.

jni · 2021-03-02T03:58:14Z

@Carreau you're exactly right (here and on twitter), metadata fixed at release time causes so many problems. 🤦

sofroniewn · 2021-03-02T04:19:40Z

I've updated our rc to just avoid 1.8.0

Carreau · 2021-03-02T04:22:42Z

@Carreau you're exactly right (here and on twitter), metadata fixed at release time causes so many problems

If ever you think you think a pep is necessary, I'm happy to contribute. If it becomes a long time problem we can try to pull resources and get funding to try to fix this upstream.

cgohlke · 2021-03-02T18:41:59Z

this has been fixed upstream, and will presumably be in the next pydantic release: pydantic/pydantic#2421

Trying pydantic-1.8.0 with the fix, napari-0.4.6 still fails to start (on Windows):

> py -m napari
Traceback (most recent call last):
  File "X:\Python38\lib\runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "X:\Python38\lib\runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "X:\Python38\lib\site-packages\napari\__main__.py", line 327, in <module>
    sys.exit(main())
  File "X:\Python38\lib\site-packages\napari\__main__.py", line 323, in main
    _run()
  File "X:\Python38\lib\site-packages\napari\__main__.py", line 249, in _run
    _viewer = view_path(  # noqa: F841
  File "X:\Python38\lib\site-packages\napari\view_layers.py", line 6, in view_path
    function signature is a merged version of one of the ``Viewer.__init__`` method
  File "X:\Python38\lib\site-packages\napari\viewer.py", line 52, in __init__
    self._window = Window(self, show=show)
  File "X:\Python38\lib\site-packages\napari\_qt\qt_main_window.py", line 251, in __init__
    self._add_view_menu()
  File "X:\Python38\lib\site-packages\napari\_qt\qt_main_window.py", line 414, in _add_view_menu
    toggle_theme.triggered.connect(self.qt_viewer.viewer._toggle_theme)
TypeError: unhashable type: 'Viewer'

Reverting to pydantic-1.7.3 works.

tlambert03 · 2021-03-02T19:07:59Z

thanks @cgohlke. I just tried with the current pydantic master and napari 0.4.6, but it worked for me on my mac (strange though... doesn't seem this would be an OS specific error). I can get on this for windows though and we can try to make sure to have a napari release that works for it when it comes out.

sofroniewn · 2021-03-02T19:08:15Z

Thanks for flagging @cgohlke - is that fixed by pydantic/pydantic#2423?

cgohlke · 2021-03-02T19:57:43Z

Yes, using pydantic master works. Sorry for the noise.

sofroniewn · 2021-03-02T20:09:15Z

Yes, using pydantic master works. Sorry for the noise.

No worries, great to have the additional confirmation! Always keep your bug reports coming :-)

sofroniewn · 2021-03-04T03:09:22Z

1.8.1 just came out, not sure if it's breaking 4.6.0, but i suspect it is breaking master due to

  E   TypeError: Object of type 'cycle' is not JSON serializable` error,

failing in the colormanger now see https://github.com/napari/napari/pull/2344/checks?check_run_id=2026313230

cc @tlambert03 @kevinyamauchi

not sure what could have happened from the changelog https://pydantic-docs.helpmanual.io/changelog/#v181-2021-03-03

sofroniewn · 2021-03-04T03:22:56Z

Ok, yup I've checked this now, it is the 1.8.1 update, i havn't tried to find the PR yet, but this is the full traceback on master right now

_____________________________________________________________________________________ test_init_color_manager_colormap _____________________________________________________________________________________

    def test_init_color_manager_colormap():
        n_colors = 10
        color_cycle = [[0, 0, 0, 1], [1, 1, 1, 1]]
        properties = {'point_type': _make_cycled_properties([0, 1.5], n_colors)}
        color_manager = ColorManager._from_layer_kwargs(
            colors='point_type',
            properties=properties,
            n_colors=n_colors,
            continuous_colormap='gray',
            contrast_limits=None,
            categorical_colormap=color_cycle,
        )

        assert len(color_manager.colors) == n_colors
        assert color_manager.color_mode == 'colormap'
        color_array = transform_color(['black', 'white'] * int(n_colors / 2))
        colors = color_manager.colors.copy()
        np.testing.assert_allclose(colors, color_array)
        np.testing.assert_allclose(color_manager.current_color, [1, 1, 1, 1])
        assert color_manager.color_properties.current_value == 1.5

        # test that colormanager state can be saved and loaded
        cm_dict = color_manager.dict()
        color_manager_2 = ColorManager._from_layer_kwargs(
            colors=cm_dict, properties=properties
        )
        assert color_manager == color_manager_2

        # test json serialization
>       json_str = color_manager.json()

napari/layers/utils/_tests/test_color_manager.py:538:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
pydantic/main.py:551: in pydantic.main.BaseModel.json
    ???
../../opt/anaconda3/lib/python3.7/json/__init__.py:238: in dumps
    **kw).encode(obj)
../../opt/anaconda3/lib/python3.7/json/encoder.py:199: in encode
    chunks = self.iterencode(o, _one_shot=True)
../../opt/anaconda3/lib/python3.7/json/encoder.py:257: in iterencode
    return _iterencode(o, 0)
pydantic/json.py:107: in pydantic.json.custom_pydantic_encoder
    ???
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

>   ???
E   TypeError: Object of type 'cycle' is not JSON serializable

pydantic/json.py:95: TypeError

sofroniewn · 2021-03-04T03:31:36Z

Oh now that i think about this, it could have broken in 1.8.0, but we just hadn't seen it yet!

See napari/napari#2322 and links at bottom.

sofroniewn added this to the 0.4.6 milestone Feb 26, 2021

sofroniewn added the bug Something isn't working label Feb 26, 2021

sofroniewn mentioned this issue Feb 26, 2021

Pin pydantic < 1.8.0 #2323

Merged

sofroniewn mentioned this issue Mar 2, 2021

Change pydantic pin #2333

Merged

sofroniewn modified the milestones: 0.4.6, 0.4.7 Mar 4, 2021

This was referenced Mar 4, 2021

Consolidate polygon base object #2344

Merged

Use not mutable fields on Viewer where appropriate #2346

Merged

tlambert03 mentioned this issue Mar 5, 2021

Fix json encoder inheritance on nested submodels for pydantic>=1.8.0 #2357

Merged

7 tasks

sofroniewn closed this as completed in #2357 Mar 5, 2021

jni added a commit to regro-cf-autotick-bot/napari-feedstock that referenced this issue Mar 8, 2021

Pin pydantic to <1.8.0

1ae7e9d

See napari/napari#2322 and links at bottom.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

pydantic 1.8 release #2322

pydantic 1.8 release #2322

sofroniewn commented Feb 26, 2021

sofroniewn commented Feb 26, 2021

Czaki commented Feb 27, 2021

sofroniewn commented Feb 27, 2021

Czaki commented Feb 27, 2021

adamltyson commented Mar 1, 2021

VolkerH commented Mar 1, 2021 •

edited

tlambert03 commented Mar 1, 2021 •

edited

Czaki commented Mar 1, 2021

tlambert03 commented Mar 1, 2021

sofroniewn commented Mar 1, 2021 •

edited

sofroniewn commented Mar 1, 2021

Carreau commented Mar 1, 2021

Carreau commented Mar 1, 2021

tlambert03 commented Mar 1, 2021

jni commented Mar 1, 2021

Carreau commented Mar 1, 2021

jni commented Mar 2, 2021

sofroniewn commented Mar 2, 2021

Carreau commented Mar 2, 2021

cgohlke commented Mar 2, 2021

tlambert03 commented Mar 2, 2021

sofroniewn commented Mar 2, 2021

cgohlke commented Mar 2, 2021

sofroniewn commented Mar 2, 2021

sofroniewn commented Mar 4, 2021

sofroniewn commented Mar 4, 2021

sofroniewn commented Mar 4, 2021

pydantic 1.8 release #2322

pydantic 1.8 release #2322

Comments

sofroniewn commented Feb 26, 2021

🐛 Bug

sofroniewn commented Feb 26, 2021

Czaki commented Feb 27, 2021

sofroniewn commented Feb 27, 2021

Czaki commented Feb 27, 2021

adamltyson commented Mar 1, 2021

VolkerH commented Mar 1, 2021 • edited

tlambert03 commented Mar 1, 2021 • edited

Czaki commented Mar 1, 2021

tlambert03 commented Mar 1, 2021

sofroniewn commented Mar 1, 2021 • edited

sofroniewn commented Mar 1, 2021

Carreau commented Mar 1, 2021

Carreau commented Mar 1, 2021

tlambert03 commented Mar 1, 2021

jni commented Mar 1, 2021

Carreau commented Mar 1, 2021

jni commented Mar 2, 2021

sofroniewn commented Mar 2, 2021

Carreau commented Mar 2, 2021

cgohlke commented Mar 2, 2021

tlambert03 commented Mar 2, 2021

sofroniewn commented Mar 2, 2021

cgohlke commented Mar 2, 2021

sofroniewn commented Mar 2, 2021

sofroniewn commented Mar 4, 2021

sofroniewn commented Mar 4, 2021

sofroniewn commented Mar 4, 2021

VolkerH commented Mar 1, 2021 •

edited

tlambert03 commented Mar 1, 2021 •

edited

sofroniewn commented Mar 1, 2021 •

edited