Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

tqdm is not serializable? #611

Closed
mishadr opened this issue Aug 14, 2023 · 4 comments
Closed

tqdm is not serializable? #611

mishadr opened this issue Aug 14, 2023 · 4 comments
Labels
Milestone

Comments

@mishadr
Copy link

mishadr commented Aug 14, 2023

dill 0.3.7
tqdm 4.66.1
When I try to serialize tqdm object:

from dill import dumps, loads
from tqdm import tqdm

b = dumps(tqdm())
print(loads(b))

I get a RecursionError:

0it [00:00, ?it/s]
Traceback (most recent call last):
  File "/home/misha/workspace/gnn_interpretation/src/experiments/tqdm_bug.py", line 5, in <module>
    print(loads(b))
  File "/home/misha/workspace/gnn_interpretation/venv/lib/python3.8/site-packages/dill/_dill.py", line 301, in loads
    return load(file, ignore, **kwds)
  File "/home/misha/workspace/gnn_interpretation/venv/lib/python3.8/site-packages/dill/_dill.py", line 287, in load
    return Unpickler(file, ignore=ignore, **kwds).load()
  File "/home/misha/workspace/gnn_interpretation/venv/lib/python3.8/site-packages/dill/_dill.py", line 442, in load
    obj = StockUnpickler.load(self)
  File "/home/misha/workspace/gnn_interpretation/venv/lib/python3.8/site-packages/tqdm/utils.py", line 139, in __getattr__
    return getattr(self._wrapped, name)
  File "/home/misha/workspace/gnn_interpretation/venv/lib/python3.8/site-packages/tqdm/utils.py", line 139, in __getattr__
    return getattr(self._wrapped, name)
  File "/home/misha/workspace/gnn_interpretation/venv/lib/python3.8/site-packages/tqdm/utils.py", line 139, in __getattr__
    return getattr(self._wrapped, name)
  [Previous line repeated 992 more times]
RecursionError: maximum recursion depth exceeded
Exception ignored in: <function tqdm.__del__ at 0x7ff5ade079d0>
Traceback (most recent call last):
  File "/home/misha/workspace/gnn_interpretation/venv/lib/python3.8/site-packages/tqdm/std.py", line 1149, in __del__
    self.close()
  File "/home/misha/workspace/gnn_interpretation/venv/lib/python3.8/site-packages/tqdm/std.py", line 1268, in close
    if self.disable:
AttributeError: 'tqdm' object has no attribute 'disable'

This could be connected to AttributeError: 'tqdm' object has no attribute '' #487 , but the issue seems to be solved.

@mmckerns
Copy link
Member

It would seem there's a recursive reference pointer of some kind. You can, at first blush, try increasing the max recursion depth, or you can try using dill.settings['recurse'] = True (which changes the recursion behavior in globals during serialization).

@mishadr
Copy link
Author

mishadr commented Aug 15, 2023

Thanks for the reply, but that doesn't help..

@mmckerns
Copy link
Member

Ok, if that's the case, thentdqm.tdqm is not serializable.

@mmckerns mmckerns added this to the dill-0.3.8 milestone Aug 19, 2023
@mmckerns
Copy link
Member

Probably the most appropriate thing to do is for the tqdm class to add a __reduce__ method or some other state-capturing method for serialization. I'm closing this, but feel free to reopen if you feel your question hasn't been answered.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants