import thunder tries to initialize all native executors

[kshitij12345]

import thunder initializes all native executors. I don't think this is desirable as initializing them may not be cheap. Except for default executors, we should load others only when user calls get_executor or get_all_executor.

lightning-thunder/thunder/__init__.py

Line 78 in 843b2c2

pythonex = extend.get_executor("python")

lightning-thunder/thunder/extend/__init__.py

Lines 382 to 388 in 843b2c2

	def get_executor(name: Any) -> None | Executor:
	ex: Executor
	for ex in get_all_executors():
	if ex.name == name:
	return ex
	return None

cc @apaz-cli @carmocca @Borda

[carmocca]

get_all_executors makes sure that all the 1st party executors are imported:

lightning-thunder/thunder/extend/__init__.py

Lines 354 to 366 in b1f4470

	# manually import all native executors to let them register themselves
	from thunder.executors import (
	apex_entropyex,
	cudnn_layernormex,
	cudnnex,
	nvfuserex,
	pythonex,
	sdpaex,
	torch_compile,
	torchex,
	transformer_engineex,
	triton_crossentropy,
	)

. This is necessary to avoid silently returning None for executors that are importable.

One simple fix could be avoiding get_executor at import time

Still in the long-term, Thunder would benefit from having a cheap way for executors to register themselves in a way that signals "I could be imported". And a different and more expensive check for "I can actually be used (dependencies are available, required hardware is available, ...)"

[mruberry]

triage review — adding a system that lets executors register themselves as "importable" would be a great extensibility point and should avoid actually importing everything at startup

[mruberry]

@ptrblck points out that importing thunder takes significantly longer than importing torch alone; this could be part of that issue

%time import torch
CPU times: user 745 ms, sys: 0 ns, total: 745 ms
Wall time: 774 ms

%time import thunder
CPU times: user 1.57 s, sys: 680 ms, total: 2.25 s
Wall time: 2.28 s

%time import numpy
CPU times: user 1 µs, sys: 1 µs, total: 2 µs
Wall time: 4.53 µs