Improve c10d::ReduceOp & torch.distributed.distributed_c10d.ReduceOp

[crcrpar]

c10d::ReduceOp is now a struct which contains an enum class of RedOptype in order to support PREMUL_SUM (premul_sum is only supported by NCCL backend).
This new reduce op type takes either a Python scalar or a Tensor and that scaling value needs to be stored somewhere while keeping the compatibility with dispatchable reduce ops (note that TorchScript compiler's support is limited) and keeping torch.distributed.ReduceOp instances enum like as possible (to name a few, allowing for __members__ and isinstance).

As the op type itself is marked experimental for now but with this requirements and the changes caused, we have to improve the API.
The question is how to have users pass a scale value and how to create ReduceOp (while before premul_sum, there was no need to create a ReduceOp instance as it was closer to Python enum).

Related:

• Reenable isinstance with torch.distributed.ReduceOp #87303
• Warning and breaking changes around ReduceOP in 1.13 #87191
• Add NCCL PreMul Sum to c10d redce ops #84243

cc @mrshenli @pritamdamania87 @zhaojuanmao @satgera @rohan-varma @gqchen @aazzolini @osalpekar @jiayisuse @H-Huang @kwen2501 @awgu @wanchaol @carmocca

[rohan-varma]

@crcrpar For clarity purposes, could you provide a snippet of what user currently has to do, the suggested improvement, and what the API will look like after?

cc @wanchaol @kwen2501 for comments.

[crcrpar]

What users have to do

Except premul_sum, users don't need to update their code. The new op requires a scaling factor thus I currently have users call torch.distributed._make_nccl_premul_sum. At the moment, the other ops don't require changes, e.g.

pytorch/test/distributed/test_c10d_nccl.py

Line 340 in 5ee5f5a

allreduce(tensors, c10d.ReduceOp.AVG)

.
An example of premul_sum usage is

pytorch/test/distributed/test_c10d_nccl.py

Line 355 in 5ee5f5a

allreduce(tensors, c10d._make_nccl_premul_sum(factor))

What users cannot do

As @wanchaol reported in #87303 (comment), copy is disalloed.
Before the PR above (and after the premul_sum merge), isinstance didn't work.

The cause of these two is that c10d::ReduceOp is now a struct which has an internal enum type of RedOpType.
The motivation was to let ReduceOp holds some user-supplied data like a scaling factor for premul_sum.

Suggested Improvement

Make ReduceOp more enum like by writing a custom __instancecheck__

API change

how we create a reduceop instance of premul_sum could change in the future.
@wanchaol suggested something like ReduceOp.PREMUL_SUM(scale) instead of forcing us to call dist._make_nccl_premul_sum.