New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[AI Accelerators] Update torch.nn.functional multi_head_attention_forward(). Add pastpath #88912
Conversation
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/88912
Note: Links to docs will display an error until the docs builds have been completed. ❌ 1 FailuresAs of commit 00b7ca7: This comment was automatically generated by Dr. CI and updates every 15 minutes. |
|
None, | ||
None, | ||
multi_head_attn_nn.in_proj_weight, | ||
multi_head_attn_nn.in_proj_bias, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What is this change? Is this to avoid the assertion that you added elsewhere to prevent bias from being None?
Projection bias can be None, see the test and fix in this as per issue raised. #88970
Looks like this PR hasn't been updated in a while so we're going to go ahead and mark this as |
Summary: It presents as a fastpath where common execution paths are executed natively in Pytorch using torch._native_multi_head_attention. At present, the fast path is implemented in torch.nn.MultiHeadAttention.
Change replicates fastpath implementation in torch.nn.functional multi_head_attention_forward.
With some refactoring done to avoid duplication.
Test Plan: buck2 test @//mode/dev-nosan //caffe2/test:jit -- 'jit_multihead_attn_forward (test_jit.TestJit)' test_functional_multi_head_attn_fast_path --print-passing-details