Add esbuild-py

[keller-mark]

🐍 Package Request

Hi, I am interested in creating a pyodide package for https://github.com/keller-mark/esbuild-py to be used in jupyterlite. Esbuild-py contains bindings to the Golang-based esbuild.

I cannot simply run

GOOS=js GOARCH=wasm go build -o _esbuild.wasm

because cgo (which i am using to create the existing non-pyodide Python bindings) is not compatible with WASM. However there is already a WASM build of esbuild available https://github.com/evanw/esbuild/blob/70893e4d89bab1dbf2d512593f9e4d5bcb1f76bf/scripts/esbuild.js#L129

I am just unsure how to construct the meta.yaml file and subsequently how to modify/extend the ctypes-based bindings https://github.com/keller-mark/esbuild-py/blob/1d3aebd2cda143b84cca9349861f44cc8d1676d4/src/esbuild_py/esbuild.py#L32 to support WASM-based bindings which will presumably work differently.

• Package Name and Version : esbuild-py
• Package URL : https://pypi.org/project/esbuild-py/
• Package Dependencies that needs to be resolved first: Go dependency https://github.com/keller-mark/esbuild-py/blob/1d3aebd2cda143b84cca9349861f44cc8d1676d4/setup.py#L13

[keller-mark]

I think the question comes down to how to create a pyodide package that contains bindings for an existing .wasm binary

[ryanking13]

That's an interesting problem.

If you can build esbuild WASM binaries with the Emscripten toolchain, I guess we could treat it like a normal Emscripten shared library and load it in Pyodide, but I doubt that WASM binaries built with go's build toolchain would be compatible.

[hoodmane]

It only seems to expose a single function which takes a JavaScript string and returns a JavaScript string. So maybe just use the JS FFI?

[keller-mark]

I doubt that WASM binaries built with go's build toolchain would be compatible

Is this due to a differing ABI between what Emscripten would produce and what the Go WASM implementation would produce? It seems that Go does not yet support exporting functions via wasm which presents another obstacle to this method. I was able to use GOOS=wasip1 GOARCH=wasm go build -o _esbuild.so but then I run into this

Python 3.11.3 (main, Mar 31 2024, 11:27:51) [Clang 18.0.0 (https://github.com/llvm/llvm-project 75501f53624de92aafce2f1da698 on emscripten
>>> so = ctypes.CDLL(lib_path)
>>> so.transform
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/lib/python311.zip/ctypes/__init__.py", line 389, in __getattr__
  File "/lib/python311.zip/ctypes/__init__.py", line 394, in __getitem__
AttributeError: Tried to lookup unknown symbol "transform" in dynamic lib: /Users/mkeller/research/dbmi/vitessce/esbuild-py/build/lib.emscripten_3_1_46_wasm32-cpython-311/_esbuild.cpython-311-wasm32-emscripten.so
>>> so.__dict__
{'_name': PosixPath('/Users/mkeller/research/dbmi/vitessce/esbuild-py/build/lib.emscripten_3_1_46_wasm32-cpython-311/_esbuild.cpython-311-wasm32-emscripten.so'), '_FuncPtr': <class 'ctypes.CDLL.__init__.<locals>._FuncPtr'>, '_handle': 23285320}

maybe just use the JS FFI?

Thanks very much for this suggestion, I will proceed with this method, calling the official esbuild WASM build that works in the browser using https://pyodide.org/en/stable/usage/api/python-api/code.html#pyodide.code.run_js