Revisit event loop blocking functions

[sk1p]

Remainders from #1572:

• unsnooze -> get_executor: slow on some machines, so it might need to be async (or can it run in a thread?) Snooze state display in UI #1578
• shutdown: closing the executor could take a while? (this might need to be refactored somehow, as we can't just make it async... and cleaning up the snooze task can't be done reliably from a different thread, which makes things difficult...)
• make_executor Snooze state display in UI #1578

Regarding unsnooze and make_executor - both are basically blocked by this issue:

LiberTEM/src/libertem/web/server.py

Lines 244 to 252 in 5d4d2de

	async def create_and_set_executor():
	if executor_spec is not None:
	# Executor creation is blocking (and slow) but we
	# need to run this within the main loop so that Distributed
	# attaches to that rather than trying to create its own, see:
	# https://github.com/LiberTEM/LiberTEM/pull/1535#pullrequestreview-1699340445
	# TL;DR: without this, our call to `loop.run_forever` causes
	# `RuntimeError: This event loop is already running`
	shared_state.create_and_set_executor(executor_spec)

Also, the cluster spin-up could be more lazy, like only wait for a fraction of workers to come up, instead of all of them, and already start servicing real user requests.

[sk1p]

• unsnooze -> get_executor: slow on some machines, so it might need to be async (or can it run in a thread?)
• shutdown: closing the executor could take a while?
• make_executor

Regarding unsnooze and make_executor - both are basically blocked by this issue:

LiberTEM/src/libertem/web/server.py

Lines 244 to 252 in 5d4d2de

	async def create_and_set_executor():
	if executor_spec is not None:
	# Executor creation is blocking (and slow) but we
	# need to run this within the main loop so that Distributed
	# attaches to that rather than trying to create its own, see:
	# https://github.com/LiberTEM/LiberTEM/pull/1535#pullrequestreview-1699340445
	# TL;DR: without this, our call to `loop.run_forever` causes
	# `RuntimeError: This event loop is already running`
	shared_state.create_and_set_executor(executor_spec)

This is now taken care of as part of #1578 - shutdown probably can't be async, as that is called too late in the whole process. shutdown is generally a bit broken right now - if it's called from a different thread than where the loop is running, we can't reliably wait for the cancellation to actually go through (and if the event loop is dead, it's pointless anyways)