RuntimeError: MPS: Unsupported Border padding mode

[bukhalmae145]

🐛 Describe the bug

python interpolate_video_forward_anyfps.py -i /Users/workstation/Movies/AFI-ForwardDeduplicate/input.mov -o /Users/workstation/Movies/AFI-ForwardDeduplicate/frames -nf 2 -fps 60 -m gmfss -s -st 12 -scale 1.0 -stf -c Loaded model 0%| | 0/1108.0 [00:00<?, ?it/s]Traceback (most recent call last): File "/Users/workstation/Movies/AFI-ForwardDeduplicate/interpolate_video_forward_anyfps.py", line 278, in <module> output0, count = decrease_inference(queue_input.copy()) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/workstation/Movies/AFI-ForwardDeduplicate/interpolate_video_forward_anyfps.py", line 207, in decrease_inference to_numpy(make_inf(inp0, inp1, scale, 0.5)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/workstation/Movies/AFI-ForwardDeduplicate/interpolate_video_forward_anyfps.py", line 190, in make_inf return model.inference(x, y, model.reuse(x, y, _scale), timestep) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/workstation/Movies/AFI-ForwardDeduplicate/models/model_pg104/GMFSS.py", line 107, in inference rife = self.ifnet(imgs, timestep, scale_list=[8, 4, 2, 1]) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/workstation/Movies/AFI-ForwardDeduplicate/Deduplication/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1527, in _wrapped_call_impl return self._call_impl(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/workstation/Movies/AFI-ForwardDeduplicate/Deduplication/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1536, in _call_impl return forward_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/workstation/Movies/AFI-ForwardDeduplicate/models/model_pg104/IFNet_HDv3.py", line 93, in forward warped_img0 = warp(img0, flow[:, :2]) ^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/workstation/Movies/AFI-ForwardDeduplicate/models/model_pg104/warplayer.py", line 22, in warp return torch.nn.functional.grid_sample(input=tenInput, grid=g, mode='bilinear', padding_mode='border', align_corners=True) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/workstation/Movies/AFI-ForwardDeduplicate/Deduplication/lib/python3.11/site-packages/torch/nn/functional.py", line 4351, in grid_sample return torch.grid_sampler(input, grid, mode_enum, padding_mode_enum, align_corners) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ RuntimeError: MPS: Unsupported Border padding mode

NotImplementedError: The operator 'aten::put_' is not currently implemented for the MPS device. If you want this op to be added in priority during the prototype phase of this feature, please comment on https://github.com/pytorch/pytorch/issues/77764. As a temporary fix, you can set the environment variable PYTORCH_ENABLE_MPS_FALLBACK=1 to use the CPU as a fallback for this op. WARNING: this will be slower than running natively on MPS.

Source Code: https://www.dropbox.com/s/dfqbbuffjmo9f32/AFI-ForwardDeduplicate.zip?dl=0

I use M1 Mac.

Versions

PyTorch version: 2.3.0a0+git07330ff
Is debug build: False
CUDA used to build PyTorch: None
ROCM used to build PyTorch: N/A

OS: macOS 14.4.1 (arm64)
GCC version: Could not collect
Clang version: 15.0.0 (clang-1500.3.9.4)
CMake version: version 3.28.3
Libc version: N/A

Python version: 3.11.9 (v3.11.9:de54cf5be3, Apr 2 2024, 07:12:50) [Clang 13.0.0 (clang-1300.0.29.30)] (64-bit runtime)
Python platform: macOS-14.4.1-arm64-arm-64bit
Is CUDA available: False
CUDA runtime version: No CUDA
CUDA_MODULE_LOADING set to: N/A
GPU models and configuration: No CUDA
Nvidia driver version: No CUDA
cuDNN version: No CUDA
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

CPU:
Apple M1 Max

Versions of relevant libraries:
[pip3] numpy==1.26.4
[pip3] torch==2.4.0.dev20240426
[pip3] torchvision==0.18.0
[conda] Could not collect

cc @kulinseth @albanD @malfet @DenisVieriu97 @jhavukainen

[jonahclarsen]

I'm getting the same error when running https://github.com/google-research/frame-interpolation on my M2 MBA with torch 2.1.2:

  File "\torch\nn\functional.py", line 4244, in grid_sample
        align_corners = False

    return torch.grid_sampler(input, grid, mode_enum, padding_mode_enum, align_corners)
           ~~~~~~~~~~~~~~~~~~ <--- HERE
RuntimeError: MPS: Unsupported Border padding mode

[jonahclarsen]

I figured it out. The solution was, before exporting the torchscript model, change the padding mode from 'border' to 'zeros'. Haven't yet seen how this affects the model outputs however.

[bukhalmae145]

I figured it out. The solution was, before exporting the torchscript model, change the padding mode from 'border' to 'zeros'. Haven't yet seen how this affects the model outputs however.

get this error if I change to 'zeros'

File "/Users/workstation/Movies/AFI-ForwardDeduplicate/models/model_pg104/forward_warp2.py", line 173, in sample_one img_warp.put_(ids_mask, torch.masked_select(flat_img*flat_weight, mask), accumulate=True) IndexError: out of range: tried to access index 33423360 on a tensor of 33423360 elements.