ValueError: weight_norm of 'weight' not found in ParametrizedConvTranspose1d

[SamuelLarkin]

🐛 Describe the bug

Hi,
I'm trying to wrap and unwrap weight normalization and I get an error. Strangely it is complaining about weight not be present where we can see it when printing the model.

I was seeing warnings.warn("torch.nn.utils.weight_norm is deprecated in favor of torch.nn.utils.parametrizations.weight_norm.") in my logs thus I decided to change the deprecated function for the suggested one and then I started to get the error. If we change from torch.nn.utils.parametrizations import weight_norm back to from torch.nn.utils import remove_weight_norm, weight_norm, we get a succes.

Code

#!/usr/bin/env  python3
# coding: utf-8

from torch.nn import ConvTranspose1d
from torch.nn.utils import remove_weight_norm
from torch.nn.utils.parametrizations import weight_norm

c = ConvTranspose1d(512, 256, kernel_size=(16,), stride=(8,), padding=(4,))
print(c)
m = weight_norm(c)
print(m)
remove_weight_norm(m)

Error

ConvTranspose1d(512, 256, kernel_size=(16,), stride=(8,), padding=(4,))
ParametrizedConvTranspose1d(
  512, 256, kernel_size=(16,), stride=(8,), padding=(4,)
  (parametrizations): ModuleDict(
    (weight): ParametrizationList(
      (0): _WeightNorm()
    )
  )
)

Traceback (most recent call last):
  File "$HOME/norm_weight_error.py", line 10, in <module>
    remove_weight_norm(m)
  File "$HOME/.conda/envs/test/lib/python3.11/site-packages/torch/nn/utils/weight_norm.py", line 153, in remove_weight_norm
    raise ValueError(f"weight_norm of '{name}' not found in {module}")
ValueError: weight_norm of 'weight' not found in ParametrizedConvTranspose1d(
  512, 256, kernel_size=(16,), stride=(8,), padding=(4,)
  (parametrizations): ModuleDict(
    (weight): ParametrizationList(
      (0): _WeightNorm()
    )
  )
)

Versions

Collecting environment information...
PyTorch version: 2.1.0+cu118
Is debug build: False
CUDA used to build PyTorch: 11.8
ROCM used to build PyTorch: N/A

OS: Ubuntu 22.04.4 LTS (x86_64)
GCC version: (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0
Clang version: 14.0.0-1ubuntu1.1
CMake version: version 3.22.1
Libc version: glibc-2.35

Python version: 3.11.7 (main, Dec 15 2023, 18:12:31) [GCC 11.2.0] (64-bit runtime)
Python platform: Linux-5.15.0-102-generic-x86_64-with-glibc2.35
Is CUDA available: False
CUDA runtime version: No CUDA
CUDA_MODULE_LOADING set to: N/A
GPU models and configuration: No CUDA
Nvidia driver version: No CUDA
cuDNN version: No CUDA
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

CPU:
Architecture: x86_64
CPU op-mode(s): 32-bit, 64-bit
Address sizes: 48 bits physical, 48 bits virtual
Byte Order: Little Endian
CPU(s): 128
On-line CPU(s) list: 0-127
Vendor ID: AuthenticAMD
Model name: AMD EPYC 7532 32-Core Processor
CPU family: 23
Model: 49
Thread(s) per core: 2
Core(s) per socket: 32
Socket(s): 2
Stepping: 0
Frequency boost: enabled
CPU max MHz: 2400.0000
CPU min MHz: 1500.0000
BogoMIPS: 4791.10
Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc rep_good
nopl nonstop_tsc cpuid extd_apicid aperfmperf rapl pni pclmulqdq monitor ssse3 fma cx16 sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misaligns
se 3dnowprefetch osvw ibs skinit wdt tce topoext perfctr_core perfctr_nb bpext perfctr_llc mwaitx cpb cat_l3 cdp_l3 hw_pstate ssbd mba ibrs ibpb stibp vmmcall fsgsbase bmi1 avx2 smep bmi2 cqm rdt_a rdsee
d adx smap clflushopt clwb sha_ni xsaveopt xsavec xgetbv1 cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local clzero irperf xsaveerptr rdpru wbnoinvd amd_ppin arat npt lbrv svm_lock nrip_save tsc_scale vmc
b_clean flushbyasid decodeassists pausefilter pfthreshold avic v_vmsave_vmload vgif v_spec_ctrl umip rdpid overflow_recov succor smca
Virtualization: AMD-V
L1d cache: 2 MiB (64 instances)
L1i cache: 2 MiB (64 instances)
L2 cache: 32 MiB (64 instances)
L3 cache: 512 MiB (32 instances)
NUMA node(s): 2
NUMA node0 CPU(s): 0-31,64-95
NUMA node1 CPU(s): 32-63,96-127
Vulnerability Gather data sampling: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Retbleed: Mitigation; untrained return thunk; SMT enabled with STIBP protection
Vulnerability Spec rstack overflow: Mitigation; safe RET
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl and seccomp
Vulnerability Spectre v1: Mitigation; usercopy/swapgs barriers and __user pointer sanitization
Vulnerability Spectre v2: Mitigation; Retpolines, IBPB conditional, STIBP always-on, RSB filling, PBRSB-eIBRS Not affected
Vulnerability Srbds: Not affected
Vulnerability Tsx async abort: Not affected

Versions of relevant libraries:
[pip3] flake8==7.0.0
[pip3] mypy==1.8.0
[pip3] mypy-extensions==1.0.0
[pip3] numpy==1.26.4
[pip3] pytorch-lightning==2.2.0.post0
[pip3] torch==2.1.0+cu118
[pip3] torchaudio==2.1.0+cu118
[pip3] torchinfo==1.8.0
[pip3] torchmetrics==1.3.1
[pip3] triton==2.1.0
[conda] numpy 1.26.4 pypi_0 pypi
[conda] pytorch-lightning 2.2.0.post0 pypi_0 pypi
[conda] torch 2.1.0+cu118 pypi_0 pypi
[conda] torchaudio 2.1.0+cu118 pypi_0 pypi
[conda] torchinfo 1.8.0 pypi_0 pypi
[conda] torchmetrics 1.3.1 pypi_0 pypi
[conda] triton 2.1.0 pypi_0 pypi

[SamuelLarkin]

The trigger for this was a deprecation message warnings.warn("torch.nn.utils.weight_norm is deprecated in favor of torch.nn.utils.parametrizations.weight_norm.").
If we look at weight_norm's documentation, it doesn't mention that we now have to use remove_parametrizations. I found out about that fact by trying to diff nn/utils/weight_norm.py nn/utils/parametrizations.py and noticing that the documentation, in the code, for weight_norm() says

* To remove the weight normalization reparametrization, use
   :func:`torch.nn.utils.parametrize.remove_parametrizations`.

Starting from torch.nn, it appears that the link for weight_norm is broken as it doesn't bring weight_norm's documentation but rather simply brings us to the end of the page. But if we search for weight_norm and then click on torch.nn.utils.parametrizations.weight_norm, we get some documentation but no mention of having to use remove_paramatrizations() like it is said in the code itself.

Using weight_norm() with remove_parametrizations() we get the expected behavior.

#!/usr/bin/env  python3
# coding: utf-8
#
from torch.nn import ConvTranspose1d
from torch.nn.utils.parametrizations import weight_norm
from torch.nn.utils.parametrize import remove_parametrizations

c = ConvTranspose1d(512, 256, kernel_size=(16,), stride=(8,), padding=(4,))
print(c)
m = weight_norm(c)
print(m)
remove_parametrizations(m, "weight")

ConvTranspose1d(512, 256, kernel_size=(16,), stride=(8,), padding=(4,))
ParametrizedConvTranspose1d(
  512, 256, kernel_size=(16,), stride=(8,), padding=(4,)
  (parametrizations): ModuleDict(
    (weight): ParametrizationList(
      (0): _WeightNorm()
    )
  )
)