Python workers: pgwasm working in local dev but failing in remote dev and deployment

[jawj]

I've been trying to connect a PG DB using pgwasm (which is based on pg8000 plus wasmsockets).

Since requirements.txt isn't supported yet, the necessary packages are bundled in a local lib folder, and I made some very small patches to pgwasm here (I added a to_py() call, and removed imports of some packages that aren't needed in my case).

This is now running nicely in the local dev environment, but failing at deploy-time for non-local dev and deploy. Is this expected? Is there anything I can do to fix it?

Code is here: https://github.com/jawj/pypgworker

I can provide DB credentials privately if useful, or any Neon DB will do.

[jawj]

The error for remote dev looks like this:

⎔ Shutting down local server...
Attaching additional modules:
┌─────────────────────────────────────────────────────────┬────────┬───────────┐
│ Name                                                    │ Type   │ Size      │
├─────────────────────────────────────────────────────────┼────────┼───────────┤
│ lib/dateutil/__init__.py                                │ python │ 0.61 KiB  │
├─────────────────────────────────────────────────────────┼────────┼───────────┤
│ lib/dateutil/_common.py                                 │ python │ 0.91 KiB  │
├─────────────────────────────────────────────────────────┼────────┼───────────┤
│ lib/dateutil/_version.py                                │ python │ 0.16 KiB  │
├─────────────────────────────────────────────────────────┼────────┼───────────┤
│ lib/dateutil/easter.py                                  │ python │ 2.62 KiB  │
├─────────────────────────────────────────────────────────┼────────┼───────────┤
│ lib/dateutil/parser/__init__.py                         │ python │ 1.72 KiB  │
├─────────────────────────────────────────────────────────┼────────┼───────────┤
│ lib/dateutil/parser/_parser.py                          │ python │ 57.42 KiB │
├─────────────────────────────────────────────────────────┼────────┼───────────┤
│ lib/dateutil/parser/isoparser.py                        │ python │ 12.92 KiB │
├─────────────────────────────────────────────────────────┼────────┼───────────┤
│ lib/dateutil/relativedelta.py                           │ python │ 24.32 KiB │
├─────────────────────────────────────────────────────────┼────────┼───────────┤
│ lib/dateutil/rrule.py                                   │ python │ 65.00 KiB │
├─────────────────────────────────────────────────────────┼────────┼───────────┤
│ lib/dateutil/tz/__init__.py                             │ python │ 0.43 KiB  │
├─────────────────────────────────────────────────────────┼────────┼───────────┤
│ lib/dateutil/tz/_common.py                              │ python │ 12.67 KiB │
├─────────────────────────────────────────────────────────┼────────┼───────────┤
│ lib/dateutil/tz/_factories.py                           │ python │ 2.51 KiB  │
├─────────────────────────────────────────────────────────┼────────┼───────────┤
│ lib/dateutil/tz/tz.py                                   │ python │ 61.38 KiB │
├─────────────────────────────────────────────────────────┼────────┼───────────┤
│ lib/dateutil/tz/win.py                                  │ python │ 12.63 KiB │
├─────────────────────────────────────────────────────────┼────────┼───────────┤
│ lib/dateutil/tzwin.py                                   │ python │ 0.06 KiB  │
├─────────────────────────────────────────────────────────┼────────┼───────────┤
│ lib/dateutil/utils.py                                   │ python │ 1.92 KiB  │
├─────────────────────────────────────────────────────────┼────────┼───────────┤
│ lib/dateutil/zoneinfo/__init__.py                       │ python │ 5.75 KiB  │
├─────────────────────────────────────────────────────────┼────────┼───────────┤
│ lib/dateutil/zoneinfo/rebuild.py                        │ python │ 2.34 KiB  │
├─────────────────────────────────────────────────────────┼────────┼───────────┤
│ lib/pgwasm/__init__.py                                  │ python │ 2.22 KiB  │
├─────────────────────────────────────────────────────────┼────────┼───────────┤
│ lib/pgwasm/converters.py                                │ python │ 20.13 KiB │
├─────────────────────────────────────────────────────────┼────────┼───────────┤
│ lib/pgwasm/core.py                                      │ python │ 26.72 KiB │
├─────────────────────────────────────────────────────────┼────────┼───────────┤
│ lib/pgwasm/dbapi.py                                     │ python │ 31.61 KiB │
├─────────────────────────────────────────────────────────┼────────┼───────────┤
│ lib/pgwasm/exceptions.py                                │ python │ 0.87 KiB  │
├─────────────────────────────────────────────────────────┼────────┼───────────┤
│ lib/python_dateutil-2.9.0.post0.dist-info/top_level.txt │ text   │ 0.01 KiB  │
├─────────────────────────────────────────────────────────┼────────┼───────────┤
│ lib/six-1.16.0.dist-info/top_level.txt                  │ text   │ 0.00 KiB  │
├─────────────────────────────────────────────────────────┼────────┼───────────┤
│ lib/six.py                                              │ python │ 33.74 KiB │
├─────────────────────────────────────────────────────────┼────────┼───────────┤
│ lib/wasmsockets/__init__.py                             │ python │ 0.00 KiB  │
├─────────────────────────────────────────────────────────┼────────┼───────────┤
│ lib/wasmsockets/client.py                               │ python │ 3.43 KiB  │
└─────────────────────────────────────────────────────────┴────────┴───────────┘
Total Upload: 384.87 KiB / gzip: 90.29 KiB
✘ [ERROR] Error on remote worker: APIError: A request to the Cloudflare API (/accounts/e324fb72d593446a640fc93a362aa813/workers/scripts/pypgworker/edge-preview) failed.

      at throwFetchError
  (/Users/george/.npm/_npx/d77349f55c2be1c0/node_modules/wrangler/wrangler-dist/cli.js:121457:18)
      at fetchResult
  (/Users/george/.npm/_npx/d77349f55c2be1c0/node_modules/wrangler/wrangler-dist/cli.js:121374:5)
      at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
      at async createPreviewToken
  (/Users/george/.npm/_npx/d77349f55c2be1c0/node_modules/wrangler/wrangler-dist/cli.js:169328:29)
      at async createWorkerPreview
  (/Users/george/.npm/_npx/d77349f55c2be1c0/node_modules/wrangler/wrangler-dist/cli.js:169349:17)
      at async start
  (/Users/george/.npm/_npx/d77349f55c2be1c0/node_modules/wrangler/wrangler-dist/cli.js:169936:34) {
    text: 'A request to the Cloudflare API
  (/accounts/e324fb72d593446a640fc93a362aa813/workers/scripts/pypgworker/edge-preview) failed.',
    notes: [
      {
        text: 'Uncaught Error: PythonError: Traceback (most recent call last):\n' +
          '  File "/session/metadata/lib/pgwasm/dbapi.py", line 18, in async_to_sync\n' +
          '    loop = asyncio.get_running_loop()\n' +
          '           ^^^^^^^^^^^^^^^^^^^^^^^^^^\n' +
          'RuntimeError: no running event loop\n' +
          '\n' +
          'During handling of the above exception, another exception occurred:\n' +
          '\n' +
          'Traceback (most recent call last):\n' +
          '  File "/lib/python312.zip/_pyodide/_base.py", line 629, in pyimport_impl\n' +
          '    res = __import__(stem, fromlist=fromlist)\n' +
          '          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n' +
          '  File "/session/metadata/entry.py", line 8, in <module>\n' +
          '    import pgwasm.dbapi\n' +
          '  File "/session/metadata/lib/pgwasm/__init__.py", line 1, in <module>\n' +
          '    from pgwasm.dbapi import connect\n' +
          '  File "/session/metadata/lib/pgwasm/dbapi.py", line 241, in <module>\n' +
          '    connect = async_to_sync(aconnect)\n' +
          '              ^^^^^^^^^^^^^^^^^^^^^^^\n' +
          '  File "/session/metadata/lib/pgwasm/dbapi.py", line 20, in async_to_sync\n' +
          '    loop = asyncio.new_event_loop()\n' +
          '           ^^^^^^^^^^^^^^^^^^^^^^^^\n' +
          '  File "/lib/python312.zip/asyncio/events.py", line 819, in new_event_loop\n' +
          '    return get_event_loop_policy().new_event_loop()\n' +
          '           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n' +
          '  File "/lib/python312.zip/asyncio/events.py", line 716, in new_event_loop\n' +
          '    return self._loop_factory()\n' +
          '           ^^^^^^^^^^^^^^^^^^^^\n' +
          '  File "/lib/python312.zip/asyncio/unix_events.py", line 64, in __init__\n' +
          '    super().__init__(selector)\n' +
          '  File "/lib/python312.zip/asyncio/selector_events.py", line 66, in __init__\n' +
          '    self._make_self_pipe()\n' +
          '  File "/lib/python312.zip/asyncio/selector_events.py", line 120, in _make_self_pipe\n' +
          '    self._ssock, self._csock = socket.socketpair()\n' +
          '                               ^^^^^^^^^^^^^^^^^^^\n' +
          '  File "/lib/python312.zip/socket.py", line 635, in socketpair\n' +
          '    lsock.listen()\n' +
          'OSError: [Errno 138] Not supported\n' +
          '\n' +
          '  at pyodide-internal:generated/pyodide.asm:20:9998 in new_error\n' +
          '  at wasm://wasm/0265dd42:1:1493017 in $wrap_exception\n' +
          '  at wasm://wasm/0265dd42:1:1493363 in $pythonexc2js\n' +
          '  at pyodide-internal:generated/pyodide.asm:20:607948 in Module._pythonexc2js\n' +
          '  at pyodide-internal:generated/pyodide.asm:20:64098 in Module.callPyObjectKwargs\n' +
          '  at pyodide-internal:generated/pyodide.asm:20:65055 in Module.callPyObject\n' +
          '  at pyodide-internal:generated/pyodide.asm:20:79132 in apply\n' +
          '  at pyodide-internal:generated/pyodide.asm:20:77371 in apply\n' +
          '  at pyodide-internal:generated/pyodide.asm:20:102862 in pyimport\n' +
          '  at pyodide:python-entrypoint-helper:32:18 in pyimportMainModule\n' +
          ' [code: 10021]'
      }
    ],
    location: undefined,
    kind: 'error',
    code: 10021
  }

But, as I said, it works fine for local dev.