General MPS op coverage tracking issue #77764
Comments
albanD
added
feature
albanD
changed the title
|
Are there any linear algebra ops not implemented in MPS that you have made custom shaders for? Any shaders I could "borrow" from your project (with full credit) and use in my own? Specifically, it would be helpful to have SVD and reverse-mode Cholesky operators. |
|
Hey, There are no custom shaders at the moment as everything we needed for the basic networks we looked at was already provided by MPS (or a set of ops in MPS). Also , required functions that are not in the hot path are simply falling back to CPU for now. It is mentioned here as this is something that is possible to be done easily within the integration. But not something that is used today. |
|
I was testing a bunch of speech synthesis and vocoder models, and found the following operators missing so far:
|
|
One vote for a CPU fallback for Is there any reason, given the unified memory architecture, that every op not implemented on Metal cannot just fall back to the CPU implementation without memory copy operations? (Based, of course, on my 10,000ft view of the architecture, which I'm sure is wildly oversimplified.) |
|
Tip for everyone: Run your script with PYTORCH_ENABLE_MPS_FALLBACK=1 which will fallback to the CPU. I'm using a custom build which merges pull request #77791 so am not sure if this is included in the current build (Edit: It's not. You need to build PyTorch yourself with the pull request or trust an online build with it). |
|
Testing with some huggingface transformers code: + 1 vote for |
|
One missing op I ran into and haven't seen mentioned yet is then and finally the forward pass through my model crashes with On |
+1 |
|
@lhoenig could you open a new separate issue for the cpu fallback failing for you? @Willian-Zhang the fallback is ONLY available if you build from source right now. It will be in the nightly build tomorrow (May 21st). |
|
Would like to add |
|
I've got a non supported op: |
|
Not supported
Code X, y = torch.rand(16, 10).to("mps"), torch.rand(16, 1).to("mps")
model = nn.Linear(10, 1).to("mps")
criterion = nn.L1Loss() # nn.KLDivLoss()
loss = criterion(model(X), y)
loss.backward()Output |
|
Trying to use affine crop from torchvision, and found the operator |
|
Trying to use MPS backend with pytorch geometric, and found the operator |
|
Found the operator 'aten::grid_sampler_2d' is not current implemented for the MPS device. |
|
Would be great to add |
|
I ran into this error with |
|
Voting for |
|
🆙 |
|
Voting for |
|
Voting for |
|
Voting for |
|
Also voting for |
|
+1 for aten::upsample_bicubic2d.out |
|
NotImplementedError: The operator 'aten::index_copy.out' is not currently implemented for the MPS device. Version: 2.4.0.dev20240515 |
|
Voting for all Conv3d related and Upsample 3d related operations |
|
Voting for: Comes up with various masking attempts in self attention modules. |
|
+1 for |
|
voting for: aten::grid_sampler_2d_backward |
|
Voting for aten::isin.Tensor_Tensor_out |
1 similar comment
|
Voting for aten::isin.Tensor_Tensor_out |
|
Also voting for aten::isin.Tensor_Tensor_out |
|
nms: |
|
I tried it multiple time in M1 Ultra I always get I hope it become better in the future |
|
Voting for |
|
I find it unbelievable that with the money apple has, they don't invest in having pytorch natively with all its operations. |
|
They would rather invest in something of their own, which is what they are
doing.
ml-explore/mlx: MLX: An array framework for Apple silicon (github.com)
<https://github.com/ml-explore/mlx>
…On Wed, May 22, 2024 at 2:05 PM Johnny ***@***.***> wrote:
I find it unbelievable that with the money apple has, they don't invest in
having pytorch natively with all its operations.
—
Reply to this email directly, view it on GitHub
<#77764 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/A4V52AFMFA5PG34PRSNAH33ZDRKDZAVCNFSM5WJJ2R42U5DIOJSWCZC7NNSXTN2JONZXKZKDN5WW2ZLOOQ5TEMJSGQYTSMRUGQYQ>
.
You are receiving this because you commented.Message ID:
***@***.***>
|
Until they have easy conversion with pytorch/jax or similar difficult. Because people usually test locally and then send to the servers to train..... |
|
Voting for 'aten::isin.Tensor_Tensor_out' as well, appreciate it! |
|
Voting for: |
|
Voting for “The operator 'aten::scatter_reduce.two_out'. Thanks NotImplementedError: The operator '**
' is not currently implemented for the MPS device |
|
voting for 'aten::upsample_bicubic2d.out'. is not currently implemented for the MPS device. |
|
@albanD hey, I’m interested in working on aten::max_unpool2d |
|
voting for aten::amp_foreach_non_finite_check_and_unscale |
and others
This issue is to have a centralized place to list and track work on adding support to new ops for the MPS backend.
MPS operators coverage matrix - The matrix covers most of the supported operators but is not exhaustive. Before you comment below, please take a look at this matrix to make sure the operator you're requesting has not been implemented in nightly. More details can be found on the readme.
There are a very large number of operators in pytorch and so they are not all implemented yet for the MPS backends as it is still in the prototype phase. We will be prioritizing adding new operators based on user feedback. If possible, please also provide link to the network or use-case where this op is getting used.
If you want to work on adding support for such op, feel free to comment below to get assigned one. Please avoid pickup up an op that is already being worked on or that already has a PR associated with it.
Link to the wiki for details on how to add these ops and example PRs.
Good First Issue:
Below is list of Ops which are good to get started to add operations to MPS backend. Please consider picking them up.
nn.Conv3Daten::_weight_norm_interfaceaten::max_unpool2daten::cummin.out,aten::cummax.outaten::upsample_linear1d.outaten::lerp.Scalar_outaten::renormNot categorized:
These are the ops which are not yet picked up and need MPS implementation.
aten::slow_conv3d_forwardaten::_ctc_lossaten::avg_pool3d.outaten::linalg_qr.outaten::multilabel_margin_loss_forwardaten::unique_dimaten::_sample_dirichletaten::_fft_r2caten::upsample_bicubic2d.outaten::linalg_inv_out_helperaten::bucketizeaten::_embedding_bagaten::_standard_gammaaten::_upsample_bicubic2d_aa.outaten::'aten::_symeig_helperaten::linalg_matrix_expaten::_nested_tensor_from_maskaten::randperm.generator_outaten::_fused_sdp_choiceaten::linalg_cholesky_exaten::scatter_reduce.two_outaten::kthvalue.valuesaten::_linalg_solve_ex.resultaten::grid_sampler_2d_backward'max_pool3d(unfinished attempt Add mps support for maxpool3d #102148)WIP:
aten::kl_div_backward(Is not needed )Implemented Ops:
Ops that have MPS backend implementations.
See MPS operators coverage matrix and the readme for more details.
deprecated list
aten::histc[MPS] Add histogram ops #96652pow.Scalar_out(@qqaatw )aten::log_sigmoid_forward(@qqaatw )aten::fmax.out(@qqaatw )aten::roll[MPS] Add roll op #95168aten::hardsigmoid(@qqaatw )aten::logit(@qqaatw )linalg_solve_triangularaten::sort.values_stable[MPS] Add support for aten::sort.values_stable for MPS backend #86750aten::remainder.Tensor_out[MPS] Add support for aten::remainder.Tensor_out for MPS backend #86806aten::hardswish[MPS] Add support for aten::hardswish for MPS backend #86807aten::nansum[MPS] Add support for aten::nansum for MPS backend #86809aten::fmod.Tensor_out[MPS] Add support for aten::fmod.Tensor_out for MPS backend #86810aten::range[MPS] Add support for aten::range for MPS backend #86990aten::argsort[MPS] Add support for aten::argsort for MPS backend #86991aten::repeat_interleave[MPS] Add support for aten::repeat_interleave for MPS backend #87219aten::median[MPS] Add support for aten::median for MPS backend #87220aten::trace[MPS] Add support for aten::trace for MPS backend #87221aten::im2col(Falling back to CPU as its mostly used in preprocessing layers)aten::_cdist_forward[MPS] Register norm_dtype_out_mps and cdist #91643aten::native_group_norm_backward(Implemented by @malfet )aten::grid_sampler_2d([MPS] Add 2d grid sampler #94273)aten::upsample_nearest1d_backward.grad_inputaten::upsample_nearest1d.outaten::repeat_interleave.self_intaten::nan_to_num.outaten::unique_consecutive[MPS] Add Unique and unique_consecutive ops. #88532torch.bincount[MPS] Add bincount support for mps #91267aten::_unique2[MPS] Add Unique and unique_consecutive ops. #88532aten::unfold[MPS] Register unfold key for MPS #91266aten::triangular_solve.X[MPS] Add triangular solve op through MPSMatrixSolveTriangular #94345aten::nonzero[MPS] Add nonzero mps support #91616aten::_index_put_impl_([MPS] Add tensor::index_put op #85672)aten::amax.out(MPS: Add amax and amin Ops with tests #79682)aten::_slow_conv2d_forward([MPS] Better error message forslow_conv2d_forward#86303)aten::eye.m_out(MPS: Eye op #78408)aten::multinomial(MPS: Add multinomial op #80760 )aten::flip([MPS] Add flip #80214)aten::equal[MPS] Add equal operator #80195aten::_local_scalar_denseaten::l1_loss_backward.grad_input([MPS] Add L1 loss test #80010)aten::glu.out([MPS] Add glu #79866)aten::linspace.outMPS: add linespace op #78570aten::arange.out[MPS] Addarange_mps_outimplementation #78789aten::adaptive_max_pool2dMPS: Add adaptive max pool2d op #78410aten::count_nonzero.dim_IntListaten::softplus.out(MPS: Fixes #78930)aten::index_add.out[MPS] Addaten::index_add.out#79935aten::normal([MPS] addaten::normal.Tensor_floataten::normal.float_Tensoraten::normal.Tensor_Tensor#80297)aten::native_layer_norm_backwardMPS: add layer_norm_backward #79189aten::logical_and.out([MPS] Add logical ops #80216)aten::frac.out([MPS] Implementfracoperator #86625)aten:: masked_select[MPS] Add support for aten::masked_select on mps (#119) #85818aten::softplus_backward.grad_input([MPS] Add softplus backward #79873)aten::slow_conv_transpose2d.out(@malfet could be due to incompatibility with torchvision)aten::signbit.out(Supportsignbitin MPS. #87214)aten::cumsum.out([MPS] Add nativecumsumimplementation #88319)aten::cumprod.outaten::expm1.out(Adding expm1 to MPS #87147)aten::bitwise_xor.Tensor_out([MPS] And native bitwise_[and|or|xor] #82307)aten::bitwise_and.Tensor_out([MPS] And native bitwise_[and|or|xor] #82307)aten::bitwise_or.Tensor_out([MPS] And native bitwise_[and|or|xor] #82307)aten::index.Tensor([MPS] Register index.Tensor_out #82507)aten::index.Tensor_out([MPS] Register index.Tensor_out #82507)Ops not supported by MPS:
Ops that will require either to use the CPU fallback system or a custom Metal kernel.
aten::lgamma.outaten::linalg_householder_productThe text was updated successfully, but these errors were encountered: