Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

OAuth2Session is not pickleable #849

Open
f-tremblay opened this issue Apr 30, 2023 · 2 comments
Open

OAuth2Session is not pickleable #849

f-tremblay opened this issue Apr 30, 2023 · 2 comments

Comments

@f-tremblay
Copy link

f-tremblay commented Apr 30, 2023

Describe the bug

The OAuth2Session class is not pickleable.
The problem is caused by the lambda function within the __init__ method.
In consequence, the class can't be used in a multiprocessing/threading context.

How to reproduce

Try to pickle the class.
And try to use the class in a multiprocessing/threading context

Expected behavior

The class should support pickling without any issue.

Additional context

Here's the exception:

concurrent.futures.process._RemoteTraceback: 
"""
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/multiprocessing/queues.py", line 244, in _feed
    obj = _ForkingPickler.dumps(obj)
  File "/usr/local/lib/python3.10/multiprocessing/reduction.py", line 51, in dumps
    cls(buf, protocol).dump(obj)
AttributeError: Can't pickle local object 'OAuth2Session.__init__.<locals>.<lambda>'
"""

Here's a wrapper that fixes the issue.
I tried 2 versions, and they all work (at least for my use case) and with python 3.10.

class OAuth2SessionPickleableV1(OAuth2Session):
    """A wrapper around OAuth2Session to make it pickleable."""

    def __init__(self, *args: Any, **kwargs: Any) -> None:
        super().__init__(*args, **kwargs)

        # This is the attribute that was not pickleable
        # `self.auth = lambda r: r` was causing the issue
        # since `lambda` is not pickleable
        self.auth = self.auth_method

    def auth_method(self, r: Any) -> Any:
        return r

    def __getstate__(self) -> Dict[str, Any]:
        """Customize pickling behavior"""
        state = self.__dict__.copy()
        return state

    def __setstate__(self, state: Dict[str, Any]) -> None:
        """Customize pickling behavior"""
        self.__dict__.update(state)
        
class OAuth2SessionPickleableV2(OAuth2Session):
    """A wrapper around OAuth2Session to make it pickleable."""

    def __getstate__(self) -> Dict[str, Any]:
        """Customize pickling behavior"""
        state = self.__dict__.copy()
        state["auth"] = None  # Remove the non-pickleable 'auth' attribute
        return state

    def __setstate__(self, state: Dict[str, Any]) -> None:
        """Customize pickling behavior"""
        self.__dict__.update(state)
        self.auth = lambda r: r  # Reassign the 'auth' attribute after unpickling


@jasoriya
Copy link

@JonathanHuot I've run into the same issue and its solvable by the suggested fix. Would you be wiling to accept a PR for the same?

@auvipy
Copy link
Contributor

auvipy commented Aug 17, 2023

you can come with a PR but that need proper unit test

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants