How to use pytorch-lightning distributed training without SLURM?

[areshytko]

❓How to use pytorch-lightning distributed training without SLURM?

Couldn't find anywhere a single note or tutorial on this.

For example I have just 2 node with 4 GPUs on each.
On each node environment variables required for Pytorch distributed communication are configured (see pytorch documentation).

Is this possible to train pytorch-lightning script in this setup and if so how?

[williamFalcon]

you can configure your own environment variables and do your own setup.

Just override LightningModule.init_ddp_connection
https://pytorch-lightning.readthedocs.io/en/latest/lightning-module.html#lightningmodule-class

(corrected)

[areshytko]

you also need to override LightningModule.init_ddp_connection