cloudpickle should import Pickler class directly from the pickle module

[jvesely]

joblib/joblib/externals/cloudpickle/compat.py

Line 13 in 0c2fa8f

from _pickle import Pickler # noqa: F401

uses _pickle module to import Pickler. However, as per python 3.8 documentation[0] this class is available directly in the pickle module.

from pickle import Pickler

works just fine.

[0] https://docs.python.org/3.8/library/pickle.html#pickle.Pickler

[jvesely]

This also fixes issues with pypy3:

    import sys
    
    
    if sys.version_info < (3, 8):
        try:
            import pickle5 as pickle  # noqa: F401
            from pickle5 import Pickler  # noqa: F401
        except ImportError:
            import pickle  # noqa: F401
            from pickle import _Pickler as Pickler  # noqa: F401
    else:
        import pickle  # noqa: F401
>       from _pickle import _Pickler as Pickler  # noqa: F401
E       ModuleNotFoundError: No module named '_pickle'

../pypy38-venv/lib/pypy3.8/site-packages/joblib/externals/cloudpickle/compat.py:13: ModuleNotFoundError

[tomMoral]

cloudpickle is vendored in joblib (an exact copy of its latest version is included in external). The issue tracker for its development is here: https://github.com/cloudpipe/cloudpickle, so you should report issues relative to it there.

That being said, I think this like this here for historical reasons. From what I understand, _pickle.Pickler is a fast C implementation that could not be customized prior to 3.8, while pickle._Pickler is a python implementation that can be slow but is customizable. That is why cloudpickle tries to import the former explicitly. The cloudpickle maintainers will be able to better tell you if doing the change you propose might have performance implication.

[jvesely]

thank you.
reported as cloudpipe/cloudpickle#458

note that the code path is specific to 3.8+, _pickle.Pickler and pickle.Pickler should be equivalent (see the above-posted issue)

[jvesely]

The issue on the cloudpickle side (cloudpipe/cloudpickle#458) has been fixed in cloudpipe/cloudpickle#469.

How long does it usually take for cloudpickle changes to propagate to joblib?