You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Gunicorn + Flask App RuntimeError: Cannot re-initialize CUDA in forked subprocess. To use CUDA with multiprocessing, you must use the 'spawn' start method
#3176
Open
danerlt opened this issue
Mar 25, 2024
· 0 comments
When I call the interface, I get an error ``RuntimeError: Cannot re-initialize CUDA in forked subprocess. To use CUDA with multiprocessing, you must use the 'spawn' start method`
When I specify the -w parameter as 1, it can run normally without reporting an error.
For the same function, I replaced the Flask framework with the FastAPI + uvicorn framework. The main_fastapi.py code is as follows:
uvicorn can run multiple workers normally without reporting any errors.
Through my investigation, I found that Gunicorn uses os.fork to create child processes. The relevant code is in gunicorn/arbiter.py
defspawn_worker(self):
self.worker_age+=1worker=self.worker_class(self.worker_age, self.pid, self.LISTENERS,
self.app, self.timeout/2.0,
self.cfg, self.log)
self.cfg.pre_fork(self, worker)
pid=os.fork()
ifpid!=0:
worker.pid=pidself.WORKERS[pid] =workerreturnpid# Do not inherit the temporary files of other workersforsiblinginself.WORKERS.values():
sibling.tmp.close()
# Process Childworker.pid=os.getpid()
try:
util._setproctitle("worker [%s]"%self.proc_name)
self.log.info("Booting worker with pid: %s", worker.pid)
self.cfg.post_fork(self, worker)
worker.init_process()
sys.exit(0)
exceptSystemExit:
raiseexceptAppImportErrorase:
self.log.debug("Exception while loading the application",
exc_info=True)
print("%s"%e, file=sys.stderr)
sys.stderr.flush()
sys.exit(self.APP_LOAD_ERROR)
exceptException:
self.log.exception("Exception in worker process")
ifnotworker.booted:
sys.exit(self.WORKER_BOOT_ERROR)
sys.exit(-1)
finally:
self.log.info("Worker exiting (pid: %s)", worker.pid)
try:
worker.tmp.close()
self.cfg.worker_exit(self, worker)
exceptException:
self.log.warning("Exception during worker exit:\n%s",
traceback.format_exc())
The uvicorn framework child processes created by the spawn method. The relevant code is in uvicorn.supervisors.multiprocess.py
classMultiprocess:
def__init__(
self,
config: Config,
target: Callable[[list[socket] |None], None],
sockets: list[socket],
) ->None:
self.config=configself.target=targetself.sockets=socketsself.processes: list[SpawnProcess] = []
self.should_exit=threading.Event()
self.pid=os.getpid()
defsignal_handler(self, sig: int, frame: FrameType|None) ->None:
""" A signal handler that is registered with the parent process. """self.should_exit.set()
defrun(self) ->None:
self.startup()
self.should_exit.wait()
self.shutdown()
defstartup(self) ->None:
message=f"Started parent process [{str(self.pid)}]"color_message="Started parent process [{}]".format(click.style(str(self.pid), fg="cyan", bold=True))
logger.info(message, extra={"color_message": color_message})
forsiginHANDLED_SIGNALS:
signal.signal(sig, self.signal_handler)
for_idxinrange(self.config.workers):
process=get_subprocess(config=self.config, target=self.target, sockets=self.sockets)
process.start()
self.processes.append(process)
defshutdown(self) ->None:
forprocessinself.processes:
process.terminate()
process.join()
message=f"Stopping parent process [{str(self.pid)}]"color_message="Stopping parent process [{}]".format(click.style(str(self.pid), fg="cyan", bold=True))
logger.info(message, extra={"color_message": color_message})
The get_subprocess method is defined in uvicorn._subprocess·py:
importmultiprocessingfrommultiprocessing.contextimportSpawnProcessmultiprocessing.allow_connection_pickling()
spawn=multiprocessing.get_context("spawn")
defget_subprocess(
config: Config,
target: Callable[..., None],
sockets: list[socket],
) ->SpawnProcess:
""" Called in the parent process, to instantiate a new child process instance. The child is not yet started at this point. * config - The Uvicorn configuration instance. * target - A callable that accepts a list of sockets. In practice this will be the `Server.run()` method. * sockets - A list of sockets to pass to the server. Sockets are bound once by the parent process, and then passed to the child processes. """# We pass across the stdin fileno, and reopen it in the child process.# This is required for some debugging environments.try:
stdin_fileno=sys.stdin.fileno()
# The `sys.stdin` can be `None`, see https://docs.python.org/3/library/sys.html#sys.__stdin__.except (AttributeError, OSError):
stdin_fileno=Nonekwargs= {
"config": config,
"target": target,
"sockets": sockets,
"stdin_fileno": stdin_fileno,
}
returnspawn.Process(target=subprocess_started, kwargs=kwargs)
Can Gunicorn support using spawn to create child processes?
The text was updated successfully, but these errors were encountered:
I have a Flask Web App, which is used for embedding operations. The code of
main.py
is as follows:我使用下面的命令启动
gunicorn
:When I call the interface, I get an error ``RuntimeError: Cannot re-initialize CUDA in forked subprocess. To use CUDA with multiprocessing, you must use the 'spawn' start method`
When I specify the
-w
parameter as1
, it can run normally without reporting an error.For the same function, I replaced the Flask framework with the FastAPI + uvicorn framework. The
main_fastapi.py
code is as follows:I start uvicorn using the following command::
uvicorn can run multiple workers normally without reporting any errors.
Through my investigation, I found that Gunicorn uses
os.fork
to create child processes. The relevant code is ingunicorn/arbiter.py
The uvicorn framework child processes created by the
spawn
method. The relevant code is inuvicorn.supervisors.multiprocess.py
The
get_subprocess
method is defined inuvicorn._subprocess·py
:Can Gunicorn support using spawn to create child processes?
The text was updated successfully, but these errors were encountered: