Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Can't pickle functions that use the colour-science library #523

Open
hotplot opened this issue Oct 25, 2023 · 1 comment
Open

Can't pickle functions that use the colour-science library #523

hotplot opened this issue Oct 25, 2023 · 1 comment

Comments

@hotplot
Copy link

hotplot commented Oct 25, 2023

When trying to cloudpickle a function that references the colour-science library, the following exception is raised:

Traceback (most recent call last):
  File "/home/hrrsxb/aip/x/test.py", line 10, in <module>
    y = cloudpickle.loads(x)
  File "/home/hrrsxb/.cache/pypoetry/virtualenvs/x-Z-zmud14-py3.10/lib/python3.10/site-packages/colour/__init__.py", line 911, in __getattr__
    return super().__getattr__(attribute)
  File "/home/hrrsxb/.cache/pypoetry/virtualenvs/x-Z-zmud14-py3.10/lib/python3.10/site-packages/colour/utilities/deprecation.py", line 361, in __getattr__
    change = self._changes.get(attribute)

    ...
    
  File "/home/hrrsxb/.cache/pypoetry/virtualenvs/x-Z-zmud14-py3.10/lib/python3.10/site-packages/colour/__init__.py", line 911, in __getattr__
    return super().__getattr__(attribute)
  File "/home/hrrsxb/.cache/pypoetry/virtualenvs/x-Z-zmud14-py3.10/lib/python3.10/site-packages/colour/utilities/deprecation.py", line 361, in __getattr__
    change = self._changes.get(attribute)
RecursionError: maximum recursion depth exceeded

Minimal code to reproduce:

import colour
import cloudpickle


def foo():
    print(colour.__version__)


x = cloudpickle.dumps(foo)
y = cloudpickle.loads(x)

The issue does not occur when using standard pickle

Versions used are:
python: 3.10.12
cloudpickle: 2.2.1 and 3.0.0 both tested
colour: 0.4.3

@ogrisel
Copy link
Contributor

ogrisel commented Oct 25, 2023

The colour module is interactively redefined at import time here:

https://github.com/colour-science/colour/blob/v0.4.3/colour/__init__.py#L967

the fake module itself is define here:

https://github.com/colour-science/colour/blob/v0.4.3/colour/__init__.py#L905

and has a generic __getattr__ to lookup attributes + some class level attributes that are defined a-posteriori when first importing the real module.

I suspect that since it's a fake module, cloudpickle is also trying to pickle it by value instead of pickling it by reference. And the way it's designed it cannot be pickled by value.

Maybe cloudpickle's heuristic to detect what module should be pickled by value could be improved.

Alternatively, colour itself might be fixed or improved to make it safe to pickle its fake module by value, for instance by defining the class attributes in the class definition itself rather than by setting them a posteriori.

@ogrisel ogrisel changed the title Can't pickle functions that use the colour library Can't pickle functions that use the colour-science library Nov 3, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants