pytorch 1.13 support #272
Comments
f-dangel
added a commit
that referenced
this issue
Oct 31, 2022
The function was removed between torch 1.12.1 and torch 1.13. Reintroducing it should fix #272.
This was referenced Oct 31, 2022
f-dangel
added a commit
that referenced
this issue
Oct 31, 2022
torch 1.13 removed some internal functionality that BackPACK relied on, see #272.
|
Hi, thanks for reporting this. We're working on a hotfix to support PyTorch 1.13. For now, please use |
f-dangel
added a commit
that referenced
this issue
Nov 2, 2022
The function was removed between torch 1.12.1 and torch 1.13. Reintroducing it should fix the import error reported in #272.
|
As an update: The test suite is broken with |
f-dangel
added a commit
that referenced
this issue
Nov 3, 2022
See #272. Waiting for pytorch/pytorch#88312 before `torch>=1.13` can be supported.
f-dangel
added a commit
that referenced
this issue
Nov 3, 2022
* [CI] Test with `torch=={1.9.0, 1.10.0}`
* [CI] Test with `torch=={1.9.0, 1.11.0}`
* [FIX] flake8
* [CI] Test with `torch=={1.9.0, 1.12.0}`
* [TEST] Replace `parameters_to_vector` by custom function
This should fix
`test_network_diag_ggn[<class
'test.converter.converter_cases._Permute'>]`
in `test/converter/test_converter.py`. Between torch 1.11.0 and torch
1.12.0, the GGN-vector products for this case became non-contiguous, and
`torch.nn.utils.convert_parameters.parameters_to_vector` stopped working
as it uses `view`.
Here is a short self-contained snippet to reproduce the issue:
```python
from torch import Tensor, permute, rand, rand_like
from torch.autograd import grad
from torch.nn import Linear, Module
from torch.nn.utils.convert_parameters import parameters_to_vector
from backpack.utils.convert_parameters import tensor_list_to_vector
class Permute(Module):
def __init__(self):
super().__init__()
self.batch_size = 3
self.in_dim = (5, 3)
out_dim = 2
self.linear = Linear(self.in_dim[-1], out_dim)
self.linear2 = Linear(self.in_dim[-2], out_dim)
def forward(self, x):
x = self.linear(x)
x = x.permute(0, 2, 1) # method permute
x = self.linear2(x)
x = permute(x, (0, 2, 1)) # function permute
return x
def input_fn(self) -> Tensor:
return rand(self.batch_size, *self.in_dim)
model = Permute()
inputs = model.input_fn()
outputs = model(inputs)
params = list(model.parameters())
grad_outputs = rand_like(outputs)
v = [rand_like(p) for p in model.parameters()]
vJ_tuple = grad(outputs, params, grad_outputs=grad_outputs)
for p, vJ in zip(params, vJ_tuple):
# all contiguous()
print(p.shape, vJ.shape)
# between 1.11.0 and 1.12.0, the vector-Jacobian product w.r.t. the second
# linear layer's weight is not contiguous anymore
print(p.is_contiguous(), vJ.is_contiguous())
vJ_vector = parameters_to_vector(vJ_tuple)
vJ_vector = tensor_list_to_vector(vJ_tuple)
```
* [REF] Use f-string and add type hints
* [REQ] Require `torch<1.13`
See #272. Waiting for
pytorch/pytorch#88312 before `torch>=1.13`
can be supported.
* [DOC] Update changelog to prepare compatibility patch
* [DOC] fix date
Co-authored-by: Felix Dangel <fdangel@tue.mpg.de>
f-dangel
added a commit
that referenced
this issue
Nov 3, 2022
* [CI] Test with `torch=={1.9.0, 1.10.0}`
* [CI] Test with `torch=={1.9.0, 1.11.0}`
* [FIX] flake8
* [CI] Test with `torch=={1.9.0, 1.12.0}`
* [TEST] Replace `parameters_to_vector` by custom function
This should fix
`test_network_diag_ggn[<class
'test.converter.converter_cases._Permute'>]`
in `test/converter/test_converter.py`. Between torch 1.11.0 and torch
1.12.0, the GGN-vector products for this case became non-contiguous, and
`torch.nn.utils.convert_parameters.parameters_to_vector` stopped working
as it uses `view`.
Here is a short self-contained snippet to reproduce the issue:
```python
from torch import Tensor, permute, rand, rand_like
from torch.autograd import grad
from torch.nn import Linear, Module
from torch.nn.utils.convert_parameters import parameters_to_vector
from backpack.utils.convert_parameters import tensor_list_to_vector
class Permute(Module):
def __init__(self):
super().__init__()
self.batch_size = 3
self.in_dim = (5, 3)
out_dim = 2
self.linear = Linear(self.in_dim[-1], out_dim)
self.linear2 = Linear(self.in_dim[-2], out_dim)
def forward(self, x):
x = self.linear(x)
x = x.permute(0, 2, 1) # method permute
x = self.linear2(x)
x = permute(x, (0, 2, 1)) # function permute
return x
def input_fn(self) -> Tensor:
return rand(self.batch_size, *self.in_dim)
model = Permute()
inputs = model.input_fn()
outputs = model(inputs)
params = list(model.parameters())
grad_outputs = rand_like(outputs)
v = [rand_like(p) for p in model.parameters()]
vJ_tuple = grad(outputs, params, grad_outputs=grad_outputs)
for p, vJ in zip(params, vJ_tuple):
# all contiguous()
print(p.shape, vJ.shape)
# between 1.11.0 and 1.12.0, the vector-Jacobian product w.r.t. the second
# linear layer's weight is not contiguous anymore
print(p.is_contiguous(), vJ.is_contiguous())
vJ_vector = parameters_to_vector(vJ_tuple)
vJ_vector = tensor_list_to_vector(vJ_tuple)
```
* [REF] Use f-string and add type hints
* [REQ] Require `torch<1.13`
See #272. Waiting for
pytorch/pytorch#88312 before `torch>=1.13`
can be supported.
* [DOC] Update changelog to prepare compatibility patch
* [DOC] fix date
Co-authored-by: Felix Dangel <fdangel@tue.mpg.de>
f-dangel
added a commit
that referenced
this issue
Dec 19, 2022
* [FIX] Copy `_grad_input_padding` from torch==1.9 The function was removed between torch 1.12.1 and torch 1.13. Reintroducing it should fix #272. * [CI] Use latest two torch releases for tests * [FIX] Ignore flake8 warning about abstract methods * [FIX] Import * [CI] Test torch from 1.9 to 1.13 * [FIX] Ignore 'zip()' without an explicit 'strict=' parameter * [REF] Make GGNvps contiguous before flattening and concatenation * [CI] Unambiguously specify tested torch versions * [REF] Import _grad_input_padding from torch for torch<1.13 * [FIX] Exception handling for Hessians of linear functions * [REF] Same `_grad_input_padding` import strategy for conv_transpose
f-dangel
added a commit
that referenced
this issue
Dec 20, 2022
* [FIX] Copy `_grad_input_padding` from torch==1.9 The function was removed between torch 1.12.1 and torch 1.13. Reintroducing it should fix #272. * [CI] Use latest two torch releases for tests * [FIX] Ignore flake8 warning about abstract methods * [FIX] Import * [CI] Test with `torch=={1.9.0, 1.12.0}` and make tests compatible (#276) * [CI] Test with `torch=={1.9.0, 1.10.0}` * [CI] Test with `torch=={1.9.0, 1.11.0}` * [FIX] flake8 * [CI] Test with `torch=={1.9.0, 1.12.0}` * [TEST] Replace `parameters_to_vector` by custom function This should fix `test_network_diag_ggn[<class 'test.converter.converter_cases._Permute'>]` in `test/converter/test_converter.py`. Between torch 1.11.0 and torch 1.12.0, the GGN-vector products for this case became non-contiguous, and `torch.nn.utils.convert_parameters.parameters_to_vector` stopped working as it uses `view`. Here is a short self-contained snippet to reproduce the issue: ```python from torch import Tensor, permute, rand, rand_like from torch.autograd import grad from torch.nn import Linear, Module from torch.nn.utils.convert_parameters import parameters_to_vector from backpack.utils.convert_parameters import tensor_list_to_vector class Permute(Module): def __init__(self): super().__init__() self.batch_size = 3 self.in_dim = (5, 3) out_dim = 2 self.linear = Linear(self.in_dim[-1], out_dim) self.linear2 = Linear(self.in_dim[-2], out_dim) def forward(self, x): x = self.linear(x) x = x.permute(0, 2, 1) # method permute x = self.linear2(x) x = permute(x, (0, 2, 1)) # function permute return x def input_fn(self) -> Tensor: return rand(self.batch_size, *self.in_dim) model = Permute() inputs = model.input_fn() outputs = model(inputs) params = list(model.parameters()) grad_outputs = rand_like(outputs) v = [rand_like(p) for p in model.parameters()] vJ_tuple = grad(outputs, params, grad_outputs=grad_outputs) for p, vJ in zip(params, vJ_tuple): # all contiguous() print(p.shape, vJ.shape) # between 1.11.0 and 1.12.0, the vector-Jacobian product w.r.t. the second # linear layer's weight is not contiguous anymore print(p.is_contiguous(), vJ.is_contiguous()) vJ_vector = parameters_to_vector(vJ_tuple) vJ_vector = tensor_list_to_vector(vJ_tuple) ``` * [REF] Use f-string and add type hints * [REQ] Require `torch<1.13` See #272. Waiting for pytorch/pytorch#88312 before `torch>=1.13` can be supported. * [DOC] Update changelog to prepare compatibility patch * [DOC] fix date Co-authored-by: Felix Dangel <fdangel@tue.mpg.de> * [CI] Test torch from 1.9 to 1.13 * [FIX] Ignore 'zip()' without an explicit 'strict=' parameter * [REF] Make GGNvps contiguous before flattening and concatenation * [CI] Unambiguously specify tested torch versions * [REF] Import _grad_input_padding from torch for torch<1.13 * [FIX] Exception handling for Hessians of linear functions * [REF] Same `_grad_input_padding` import strategy for conv_transpose * [FIX] Merge conflict * [CI] Ignore docstring check of _grad_input_padding * [DOC] Add type annotation, remove unused import * [DOC] Add type annotation for output
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
It seems like this function was removed between 1.12.1 and 1.13.
The text was updated successfully, but these errors were encountered: