nogil + cloudpickle cannot capture imported modules

[YangZhou1997]

Hi

I found that under the nogil python, cloudpickle can no longer capture any imported modules for functions. For example, in the code below, unpickeld_fn() will give "NameError: name 'sys' is not defined" error, while gil python would perform normally. Any suggestions or comments on how to fix this issue?

import sys
import cloudpickle
import pickle

def fn():
  print(sys.path)
  
# this works
fn()

pickeld_fn = cloudpickle.dumps(fn)
unpickeld_fn = pickle.loads(pickeld_fn)

# but this will give "NameError: name 'sys' is not defined" error
unpickeld_fn()

I am using Python 3.9.10 nogil, cloudpickle 2.2.1 (which has merged this patch).

Btw, the nogil project is really impressive!

Best,
Yang

[colesbury]

Hi @YangZhou1997, thanks for the bug report. I'm able to reproduce the issue from nogil installed via pyenv, but not on the most recent builds. I guess I must have fixed something in nogil Python since then.

I'll work on getting a new version on pyenv. That may take a little while.

What OS are you using? In the meantime you can try:

On Linux, you can build from source or use the docker image "nogil/python".
On macOS and Windows, you can download a more recent build from https://github.com/colesbury/nogil/releases (or build from source).

[colesbury]

I've put up a PR on pyenv to add a more recent build pyenv/pyenv#2602

[YangZhou1997]

Hi @colesbury, I am using Ubuntu20.04 with kennel 5.4.0. I just build from source and now the cloudpickle example works. Thanks for quickly resolving this issue!

[colesbury]

Hi @YangZhou1997, the new version (nogil-3.9.10-1) is on pyenv if you prefer that going forward.

pyenv update
pyenv install nogil-3.9.10-1

(I think it may take a bit longer to be available on macOS, where pyenv is typically installed via homebrew.)