Add python support for Functions Emulator.

[taeold]

Add support for loading and serving functions written using the Firebase Functions Python SDK (WIP: https://github.com/firebase/firebase-functions-python)

This PR is a fork of #4653 extended with support for emulating Python functions.

Python Runtime Delegate implementation is unsurprising but does include some additional wrapper code to make sure all commands (e.g. spinning up the admin server) runs within the virtualenv environment. For now, we hardcode virtual environment venv directory exists on the developer's laptop, but we'll later add support for specifying arbitrary directory for specifying virtualenv directory via firebase.json configuration.

Another note is that each emulated Python function will bind to a Port instead of Unix Domain Socket (UDS) as done when emulating Node.js function. This is because there is no straightfoward, platform-neutral way to bind python webserver to UDS. Finding large number of open port might have a bit more performance penalty and cause bugs due to race condition (similar to #5418) but it seems that we have no other choice atm.

[taeold]

Current dependencies on/for this PR:

• master
  • PR Remove go discovery module for CF3 #5419
    • PR Refactor how node binary is retrieved for functions/extensions. #5422
      • PR Add python support for Functions Emulator. #5423 👈
        • PR Enforce function timeout at worker level. #5464
          • PR Add option to initialize python functions. #5465
            • PR Fix emulator support for cloudevent to better conform to production specification. #5466
        • PR Add integration test for python support for functions emulator. #5462

This comment was auto-generated by Graphite.

[codecov-commenter]

Codecov Report

Base: 56.40% // Head: 56.30% // Decreases project coverage by -0.11% ⚠️

Coverage data is based on head (b5b5fc8) compared to base (9c61285).
Patch coverage: 33.89% of modified lines in pull request are covered.

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #5423      +/-   ##
==========================================
- Coverage   56.40%   56.30%   -0.11%     
==========================================
  Files         313      315       +2     
  Lines       21245    21348     +103     
  Branches     4336     4353      +17     
==========================================
+ Hits        11984    12020      +36     
- Misses       8221     8287      +66     
- Partials     1040     1041       +1     

Impacted Files	Coverage Δ
src/emulator/functionsRuntimeWorker.ts	64.63% <0.00%> (-0.40%)	⬇️
src/emulator/functionsEmulator.ts	8.09% <25.92%> (+0.82%)	⬆️
src/deploy/functions/runtimes/python/index.ts	33.80% <33.80%> (ø)
src/functions/python.ts	41.66% <41.66%> (ø)
src/deploy/functions/runtimes/discovery/index.ts	87.23% <100.00%> (ø)
src/deploy/functions/runtimes/index.ts	56.00% <100.00%> (+1.83%)	⬆️

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.

[pr-Mais]

UPDATE: it's working now, npm wasn't linked to the correct repository.

---

Hi Daniel! thanks for your work
I merged this branch with #4653 to test out the emulator, I assume you aren't done yet or maybe I'm missing out on a step cause I couldn't get it to work, following is what I did. I think it thinks it's NodeJS.

I created a new functions project with Python, installed emulators, then: firebase emulators:start, but got the following error:

[debug] [2023-01-18T13:37:27.463Z] Error: Cannot find module '/Users/mais/Invertase/firebase-functions-python2/samples/basic_https/functions/package.json'

Full logs:

[debug] [2023-01-18T13:37:25.228Z] ----------------------------------------------------------------------
[debug] [2023-01-18T13:37:25.230Z] Command:       /Users/mais/.nvm/versions/node/v14.21.1/bin/node /Users/mais/.nvm/versions/node/v14.21.1/bin/firebase emulators:start
[debug] [2023-01-18T13:37:25.230Z] CLI Version:   11.20.0
[debug] [2023-01-18T13:37:25.230Z] Platform:      darwin
[debug] [2023-01-18T13:37:25.230Z] Node Version:  v14.21.1
[debug] [2023-01-18T13:37:25.231Z] Time:          Wed Jan 18 2023 16:37:25 GMT+0300 (Arabian Standard Time)
[debug] [2023-01-18T13:37:25.232Z] ----------------------------------------------------------------------
[debug] 
[debug] [2023-01-18T13:37:25.282Z] > command requires scopes: ["email","openid","https://www.googleapis.com/auth/cloudplatformprojects.readonly","https://www.googleapis.com/auth/firebase","https://www.googleapis.com/auth/cloud-platform"]
[debug] [2023-01-18T13:37:25.282Z] > authorizing via signed-in user (mais@invertase.io)
[info] i  emulators: Starting emulators: functions {"metadata":{"emulator":{"name":"hub"},"message":"Starting emulators: functions"}}
[debug] [2023-01-18T13:37:26.546Z] [logging] Logging Emulator only supports listening on one address (127.0.0.1). Not listening on ::1
[debug] [2023-01-18T13:37:26.547Z] assigned listening specs for emulators {"user":{"hub":[{"address":"127.0.0.1","family":"IPv4","port":4400},{"address":"::1","family":"IPv6","port":4400}],"ui":[{"address":"127.0.0.1","family":"IPv4","port":4000},{"address":"::1","family":"IPv6","port":4000}],"logging":[{"address":"127.0.0.1","family":"IPv4","port":4500}]},"metadata":{"message":"assigned listening specs for emulators"}}
[debug] [2023-01-18T13:37:26.551Z] [hub] writing locator at /var/folders/fs/kl7rgnx14yl5wkv3mv_jtngh0000gn/T/hub-python-functions-testing.json
[debug] [2023-01-18T13:37:27.455Z] [functions] Functions Emulator only supports listening on one address (127.0.0.1). Not listening on ::1
[debug] [2023-01-18T13:37:27.456Z] [eventarc] Eventarc Emulator only supports listening on one address (127.0.0.1). Not listening on ::1
[debug] [2023-01-18T13:37:27.456Z] late-assigned ports for functions and eventarc emulators {"user":{"hub":[{"address":"127.0.0.1","family":"IPv4","port":4400},{"address":"::1","family":"IPv6","port":4400}],"ui":[{"address":"127.0.0.1","family":"IPv4","port":4000},{"address":"::1","family":"IPv6","port":4000}],"logging":[{"address":"127.0.0.1","family":"IPv4","port":4500}],"functions":[{"address":"127.0.0.1","family":"IPv4","port":5001}],"eventarc":[{"address":"127.0.0.1","family":"IPv4","port":9299}]},"metadata":{"message":"late-assigned ports for functions and eventarc emulators"}}
[warn] ⚠  functions: The following emulators are not running, calls to these services from the Functions emulator will affect production: auth, firestore, database, hosting, pubsub, storage {"metadata":{"emulator":{"name":"functions"},"message":"The following emulators are not running, calls to these services from the Functions emulator will affect production: \u001b[1mauth, firestore, database, hosting, pubsub, storage\u001b[22m"}}
[info] i  emulators: Shutting down emulators. {"metadata":{"emulator":{"name":"hub"},"message":"Shutting down emulators."}}
[info] i  functions: Stopping Functions Emulator {"metadata":{"emulator":{"name":"functions"},"message":"Stopping Functions Emulator"}}
[info] i  hub: Stopping emulator hub {"metadata":{"emulator":{"name":"hub"},"message":"Stopping emulator hub"}}
[debug] [2023-01-18T13:37:27.463Z] Error: Cannot find module '/Users/mais/Invertase/firebase-functions-python2/samples/basic_https/functions/package.json'
Require stack:
- /Users/mais/Invertase/firebase-tools/lib/emulator/functionsEmulator.js
- /Users/mais/Invertase/firebase-tools/lib/emulator/controller.js
- /Users/mais/Invertase/firebase-tools/lib/emulator/commandUtils.js
- /Users/mais/Invertase/firebase-tools/lib/commands/database-get.js
- /Users/mais/Invertase/firebase-tools/lib/commands/index.js
- /Users/mais/Invertase/firebase-tools/lib/index.js
- /Users/mais/Invertase/firebase-tools/lib/bin/firebase.js
    at Function.Module._resolveFilename (internal/modules/cjs/loader.js:902:15)
    at Function.Module._load (internal/modules/cjs/loader.js:746:27)
    at Module.require (internal/modules/cjs/loader.js:974:19)
    at require (internal/modules/cjs/helpers.js:101:18)
    at FunctionsEmulator.getNodeBinary (/Users/mais/Invertase/firebase-tools/lib/emulator/functionsEmulator.js:636:21)
    at FunctionsEmulator.start (/Users/mais/Invertase/firebase-tools/lib/emulator/functionsEmulator.js:192:39)
    at Function.start (/Users/mais/Invertase/firebase-tools/lib/emulator/registry.js:18:24)
    at startEmulator (/Users/mais/Invertase/firebase-tools/lib/emulator/controller.js:252:44)
    at Object.startAll (/Users/mais/Invertase/firebase-tools/lib/emulator/controller.js:349:15)
    at processTicksAndRejections (internal/process/task_queues.js:95:5)
    at async /Users/mais/Invertase/firebase-tools/lib/commands/emulators-start.js:32:43
[error] 
[error] Error: An unexpected error has occurred.