Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unpickling child class fails if base class __init_subclass__ expects an argument #527

Open
Gollum999 opened this issue Nov 28, 2023 · 2 comments

Comments

@Gollum999
Copy link

Gollum999 commented Nov 28, 2023

If you have a base class that defines __init_subclass__ with an argument, unpickling a child class will fail because cloudpickle does not pass along the original kwarg when re-creating the class.

This case seems to work fine in the built-in pickle package.

This usage mirrors examples shown in the docs and the original PEP that introduced the __init_subclass__ method.

Python 3.11.4
cloudpickle 2.2.1
EDIT: This also appears to be broken with cloudpickle 3.0.0.

Example:

#!/usr/bin/env python3
import pickle
import cloudpickle
print(cloudpickle.__version__)


class Base:
    def __init_subclass__(cls, /, arg, **kwargs):
        super().__init_subclass__(**kwargs)
        cls.arg = arg


class Child(Base, arg='hello'):
    pass


print(pickle.loads(pickle.dumps(Base)))
print(pickle.loads(pickle.dumps(Child)))
print(cloudpickle.loads(cloudpickle.dumps(Base)))
print(cloudpickle.loads(cloudpickle.dumps(Child)))
2.2.1
<class '__main__.Base'>
<class '__main__.Child'>
<class '__main__.Base'>
Traceback (most recent call last):
  File "/home/tsanders/test_cloudpickle_bug.py", line 20, in <module>
    print(cloudpickle.loads(cloudpickle.dumps(Child)))
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/envs/production-3.11/lib/python3.11/site-packages/cloudpickle/cloudpickle.py", line 827, in _make_skeleton_class
    skeleton_class = types.new_class(
                     ^^^^^^^^^^^^^^^^
  File "/opt/conda/envs/production-3.11/lib/python3.11/types.py", line 75, in new_class
    return meta(name, resolved_bases, ns, **kwds)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: Base.__init_subclass__() missing 1 required positional argument: 'arg'
@kentropy
Copy link

I also encountered this issue in Ray that uses cloudpickle to serialise/deserialise ray-project/ray#42823 (comment)

@ogrisel
Copy link
Contributor

ogrisel commented Apr 9, 2024

Reading https://docs.python.org/3/reference/datamodel.html#object.__init_subclass__ it sounds that this is indeed a bug in cloudpickle but I am not familiar enough with Python internals to know how to fix it right away.

Feel free to investigate yourself and submit a PR.

EDIT: maybe cloudpickle.cloudpickle._class_getnewargs should be made aware of what is returned by obj.__getnewargs_ex__ if this method is defined on the class instance to pickle. However I don't see how that could help pickling a class definition passed to cloudpickle.dumps without passing an instance. It seems that __getnewargs_ex__ is not meant to be a classmethod.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants