Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add test for torchscripting nn.TransformerEncoder, including fast path #79796

Closed
wants to merge 2 commits into from
Closed

Add test for torchscripting nn.TransformerEncoder, including fast path #79796

wants to merge 2 commits into from
+350 −0

Commits on Jun 17, 2022

  1. Add numerical test comparing BetterDecoder and fairseq decoder (pytor… 

    …ch#79576)

Summary:
Pull Request resolved: pytorch#79576

Includes both forced and incremental decoding.

Add a new file test_transformers.py to put transformers tests in and move away from huge monolithic test_nn.py. A todo item is to move existing transformer tests from test_nn.py to test_transformers.py.

Add a numerical test comparing torch.nn._transformer_decoder_layer_fwd and fairseq decoder. Both decoders use the weights of a common nn.TransformerEncoder.

Test Plan:
```
buck build --show-output mode/opt -c fbcode.enable_gpu_sections=true -c fbcode.nvcc_arch=a100 mode/inplace  //caffe2/test:transformers

./fbcode/buck-out/gen/caffe2/test/transformers#binary.par
```
Test runs and passes!

Differential Revision: D37157391

fbshipit-source-id: 3b3f1c7fdac8269278982e0dcc2d32ff6b63547d
    @erichan1 @facebook-github-bot
    erichan1 authored and facebook-github-bot committed Jun 17, 2022
    Configuration menu
    Copy the full SHA
    e48dd57 View commit details
    Browse the repository at this point in the history

  2. Add test for torchscripting nn.TransformerEncoder, including fast path 

    Summary:
Add test just to check if TransformerEncoder will crash when enumerating over params [with_no_grad, use_torchscript, training].

Motivation for this was that TransformerEncoder fast path (so with_no_grad=True) and use_torchscript=True would crash with the issue that NestedTensor doesn't have size. This was caused because the TransformerEncoder fast path generates a NestedTensor automatically as a perf optimization and torchscript attempts to find intermediate tensor sizes while it optimizes. But NestedTensor has not implemented a size method, so things fail.

This test goes together with this fix pytorch#79480

Test Plan:
```
buck build --show-output mode/opt -c fbcode.enable_gpu_sections=true -c fbcode.nvcc_arch=a100 mode/inplace  //caffe2/test:transformers

./fbcode/buck-out/gen/caffe2/test/transformers#binary.par
```
Test runs and passes together with the changes from the PR above (I made another diff on top of this with those changes). Does not pass without the fix.

Reviewed By: mikekgfb

Differential Revision: D37222923

fbshipit-source-id: 670c58a8570b7bf459c6aeb1f11800de0dba6584
    @erichan1 @facebook-github-bot
    erichan1 authored and facebook-github-bot committed Jun 17, 2022
    Configuration menu
    Copy the full SHA
    6d647a8 View commit details
    Browse the repository at this point in the history