Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[AI Accelerators] Update torch.nn.functional multi_head_attention_forward(). Add pastpath #88912

Closed
wants to merge 22 commits into from

Conversation

ynonaolga
Copy link

Summary: It presents as a fastpath where common execution paths are executed natively in Pytorch using torch._native_multi_head_attention. At present, the fast path is implemented in torch.nn.MultiHeadAttention.
Change replicates fastpath implementation in torch.nn.functional multi_head_attention_forward.
With some refactoring done to avoid duplication.

Test Plan: buck2 test @//mode/dev-nosan //caffe2/test:jit -- 'jit_multihead_attn_forward (test_jit.TestJit)' test_functional_multi_head_attn_fast_path --print-passing-details

@pytorch-bot
Copy link

pytorch-bot bot commented Nov 11, 2022

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/88912

Note: Links to docs will display an error until the docs builds have been completed.

❌ 1 Failures

As of commit 00b7ca7:

The following jobs have failed:

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@linux-foundation-easycla
Copy link

linux-foundation-easycla bot commented Nov 11, 2022

CLA Missing ID CLA Not Signed

None,
None,
multi_head_attn_nn.in_proj_weight,
multi_head_attn_nn.in_proj_bias,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is this change? Is this to avoid the assertion that you added elsewhere to prevent bias from being None?

Projection bias can be None, see the test and fix in this as per issue raised. #88970

@github-actions
Copy link

Looks like this PR hasn't been updated in a while so we're going to go ahead and mark this as Stale.
Feel free to remove the Stale label if you feel this was a mistake.
If you are unable to remove the Stale label please contact a maintainer in order to do so.
If you want the bot to never mark this PR stale again, add the no-stale label.
Stale pull requests will automatically be closed after 30 days of inactivity.

@github-actions github-actions bot added the Stale label Jan 16, 2023
@github-actions github-actions bot closed this Feb 15, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants