Document weird behavior of runPython locals parameter

[ytetsuro]

🐛 Bug

The error occurs when you try to call other functions added to the global scope in a function.

To Reproduce

code example

const { loadPyodide } = require("pyodide");

async function hello_python() {
  let pyodide = await loadPyodide();
  return pyodide.runPython(`
def a(b):
    c(b)

def c(st):
    print(st)


a(string)
`, {locals: pyodide.toPy({
  string: `print('Hello, world!')`
})});
}

hello_python().then((res) => {console.log(res);})  

Error

  File "<exec>", line 9, in <module>
  File "<exec>", line 3, in a
NameError: name 'c' is not defined

    at new_error (/pyodide/node_modules/pyodide/pyodide.asm.js:9:10014)
    at wasm://wasm/02257296:wasm-function[300]:0x158e62
    at wasm://wasm/02257296:wasm-function[301]:0x158f69
    at Module._pythonexc2js (/pyodide/node_modules/pyodide/pyodide.asm.js:9:608600)
    at Module.callPyObjectKwargs (/pyodide/node_modules/pyodide/pyodide.asm.js:9:64408)
    at Proxy.callKwargs (/pyodide/node_modules/pyodide/pyodide.asm.js:9:79802)
    at Object.runPython (/pyodide/node_modules/pyodide/pyodide.asm.js:9:102102)
    at hello_python (/Users/yoshikawa/Desktop/workspace/web/dev/study/pyodide/index.js:5:18) {
  type: 'NameError',
  __error_address: 17863600
}

Expected behavior

output a.

Environment

• Pyodide Version : 0.25.0
• Node.js version : v20.8.1

Additional context

[ryanking13]

Thanks for the report @ytetsuro! We use the eval() function when executing python codes and this bug seems to be related to the behavior of eval().

The locals dictionary passed to runPython will be passed to the locals parameter of the eval() function. If the locals parameter is passed, the code executed by runPython (compiled and run by eval() eventually) will run in that locals namespace, so the defined function will be stored in the local scope instead of the global scope.

An interesting behavior of eval() is that, unlike Python's nested function scope, eval()'s local scope does not have access to variables defined in the parent scope. So, the function c is not visible to a.

The following code is a minimal reproducer that raises the same error in native Python:

code = compile('''
def a(s):
    c(s)
def c(s):
    print(s)
a('hello world')
''', '<string>', 'exec')

eval(code, None, {})

Anyway, I agree that this is counterintuitive, and we probably need to think of a workaround.

[hoodmane]

I resisted adding a locals argument to runPython for a long time because the behavior of exec's locals is weird. It is explicitly for executing class bodies. The locals argument is specifically intended for executing class bodies, which have perplexing visibility rules. This is one of Python's bigger design warts IMO.

The following prints "hi":

class A:
   def a(): 
      print("hi")
   
   a()

whereas this throws NameError: name 'a' is not defined. Did you mean: 'A'?:

class A:
   def a():
      print("hi")

   def b():
      a()

   b()

This second example is equivalent to your code.

[Hanae1963]

Y

[ytetsuro]

@ryanking13 @hoodmane

Thanks, Response!
I am not familiar with Python and did not understand this behavior.

I see that it is the proper behavior! Thanks for letting me know!

[hoodmane]

I am not familiar with Python and did not understand this behavior.

Even for people familiar with Python, this is pretty weird. We should consider adding a warning about it in the docs...

[hoodmane]

I think I'll reopen this and label it as a documentation issue,