Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bytes length not a multiple of item size #631

Open
tHappo opened this issue Nov 5, 2023 · 0 comments
Open

bytes length not a multiple of item size #631

tHappo opened this issue Nov 5, 2023 · 0 comments

Comments

@tHappo
Copy link

tHappo commented Nov 5, 2023

i process the .pkl by python 3.11 on windows and load it by python 3.8 on Linux

Cell In[8], line 318, in prepare_dataloaders(opt, device)
316 def prepare_dataloaders(opt, device):
317 batch_size = opt.batch_size
--> 318 data = pickle.load(open(opt.data_pkl, 'rb'))
320 opt.max_token_seq_len = data['settings'].max_len
321 opt.src_pad_idx = data['vocab']['src'].vocab.stoi[PAD_WORD]

File /usr/local/miniconda3/lib/python3.8/site-packages/dill/_dill.py:287, in load(file, ignore, **kwds)
281 def load(file, ignore=None, **kwds):
282 """
283 Unpickle an object from a file.
284
285 See :func:loads for keyword arguments.
286 """
--> 287 return Unpickler(file, ignore=ignore, **kwds).load()

File /usr/local/miniconda3/lib/python3.8/site-packages/dill/_dill.py:442, in Unpickler.load(self)
441 def load(self): #NOTE: if settings change, need to update attributes
--> 442 obj = StockUnpickler.load(self)
443 if type(obj).module == getattr(_main_module, 'name', 'main'):
444 if not self._ignore:
445 # point obj class to main

File /usr/local/miniconda3/lib/python3.8/site-packages/preshed/bloom.pyx:133, in preshed.bloom.unpickle_bloom()

File /usr/local/miniconda3/lib/python3.8/site-packages/preshed/bloom.pyx:52, in preshed.bloom.BloomFilter.from_bytes()

File /usr/local/miniconda3/lib/python3.8/site-packages/preshed/bloom.pyx:73, in preshed.bloom.bloom_from_bytes()

ValueError: bytes length not a multiple of item size

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant