Incorrect deserialization of subclasses, module changed to types

[simon-mo]

This issue is similar to #440 but I have verified it still happens after the fix (on latest master).

Somehow the deserialized subclass has __module__ of types instead of __main__. This also happen when the classes are moved to their separate files.

See the following repro script:

import cloudpickle
import multiprocessing as mp

print(cloudpickle.__version__)

class Parent:
    pass

class Child(Parent):
    pass

def get_mro(klass):
    return [f"{base.__module__}.{base.__qualname__}" for base in klass.mro()]

def task(b: bytes):
    cls = cloudpickle.loads(b)
    return str(cls), get_mro(cls)

for klass in [Parent, Child]:
    with mp.Pool() as pool:
        cls_name, mros = pool.apply(task, (cloudpickle.dumps(klass),))
    print()
    print("local class name", str(klass))
    print("deserialized class name", cls_name)
    print()
    print("local mro", get_mro(klass))
    print("deserialized mro", mros)

My output on Python 3.7 f758eb3

2.1.0.dev0

local class name <class '__main__.Parent'>
deserialized class name <class '__main__.Parent'>

local mro ['__main__.Parent', 'builtins.object']
deserialized mro ['__main__.Parent', 'builtins.object']

local class name <class '__main__.Child'>
deserialized class name <class 'types.Child'>

local mro ['__main__.Child', '__main__.Parent', 'builtins.object']
deserialized mro ['types.Child', '__main__.Parent', 'builtins.object']

[ogrisel]

Indeed I confirm the __module__ of the subclass is wrongly assigned to types also on Python 3.10.

[ender-wieczorek]

This happens because we don't save attributes which have the same value in the parent class (see the code here). In particular, this means we don't save __module__ when the child class is in the same module. When we reconstruct the class with types.new_class(), __module__ gets set to types.

I currently patch cloudpickle with:

cloudpickle.cloudpickle_fast._extract_class_dict = lambda cls: dict(cls.__dict__)

[ogrisel]

Would you mind submitting a PR with a non-regression test?

[pierreglaser]

Indeed, this issue was not fixed (knowingly) by #448, see #448 (comment). I'll get it done ASAP.