Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Basic Attention attribution #148

Merged
merged 35 commits into from Jan 16, 2023
Merged

Basic Attention attribution #148

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
35 commits
Select commit Hold shift + click to select a range
b9ccbf2
added jupyterlab dependency (for easier testing)
lsickert Sep 23, 2022
08d5dbd
initial commit attention methods, added output_attentions parameter t…
lsickert Oct 17, 2022
57ae54a
Merge branch 'main' into attention-attribution
lsickert Oct 19, 2022
544321c
added basic attention method stubs\n added attention method registry
lsickert Oct 24, 2022
d0e859f
reverted changes to output generation (forward pass done inside attri…
lsickert Nov 21, 2022
a2a2021
first working version of basic attention methods
lsickert Nov 21, 2022
7b3c4fd
fixed rounding of values in cli output
lsickert Nov 22, 2022
13fc9f3
added documentation to most methods and generalized functions
lsickert Nov 22, 2022
bb13036
Merge branch 'main' into attention-attribution\n\nNeeded to downgrade…
lsickert Nov 23, 2022
9340343
removed python 3.11 build target
lsickert Nov 24, 2022
3cfd706
fix safety warnings
lsickert Nov 24, 2022
2765d63
set correct python version in pyproject.toml
lsickert Nov 25, 2022
4dd442f
regenerated requirements without 3.11
lsickert Nov 25, 2022
b14407c
Merge branch 'main' into attention-attribution, quick fix for mps issue
lsickert Dec 9, 2022
6a72166
Merge branch 'main' into attention-attribution
lsickert Dec 12, 2022
6535b09
merge branch 'main' into attention-attribution
lsickert Jan 2, 2023
624435e
update deps after merge
lsickert Jan 2, 2023
06f89a8
include 3.11 as build target
lsickert Jan 2, 2023
7bcbe92
fix different attribution_step argument formatting
lsickert Jan 2, 2023
b2fc73c
added basic decoder-only support
lsickert Jan 2, 2023
b044b4c
fixed output error for decoder only models
lsickert Jan 3, 2023
8c344b7
removed unnecessary convergence delta references in attention attribu…
lsickert Jan 3, 2023
f51cf25
allow negative indices when selecting a specific attention head for a…
lsickert Jan 4, 2023
c6a9e70
added missing negation to head checking
lsickert Jan 4, 2023
6c9cfae
fixed last_layer_attention attribution
lsickert Jan 4, 2023
b78bcc1
use custom format_attribute_args function for attention methods
lsickert Jan 9, 2023
d27f1c3
always use decoder_input_embeds in forward output
lsickert Jan 9, 2023
cacaa31
reworked LastLayerAttention to work with any single layer and allow a…
lsickert Jan 9, 2023
a8d5264
Minor bugfixes and version bumps
gsarti Jan 10, 2023
966f63c
Generalized attention attribution
gsarti Jan 10, 2023
1301a02
updated documentation and added 'min' aggregation function
lsickert Jan 13, 2023
914ee8f
Tests, typing fix, additional checks
gsarti Jan 14, 2023
7c825ad
Fix style
gsarti Jan 14, 2023
f6f0a64
added tests for attention utils
lsickert Jan 15, 2023
f40f63b
classmethod -> staticmethod where possible
gsarti Jan 16, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/build.yml
View file
Open in desktop
Expand Up @@ -12,7 +12,7 @@ jobs:
if: github.actor != 'dependabot[bot]' && github.actor != 'dependabot-preview[bot]'
strategy:
matrix:
python-version: ["3.8", "3.9", "3.10", "3.11"]
python-version: ["3.8", "3.9", "3.10"]

steps:
- uses: actions/checkout@v3
Expand Down
3 changes: 1 addition & 2 deletions pyproject.toml
View file
Open in desktop
Expand Up @@ -35,8 +35,7 @@ classifiers = [ #! Update me
"Programming Language :: Python :: 3",
"Programming Language :: Python :: 3.8",
"Programming Language :: Python :: 3.9",
"Programming Language :: Python :: 3.10",
"Programming Language :: Python :: 3.11",
"Programming Language :: Python :: 3.10"
lsickert marked this conversation as resolved.
Show resolved Hide resolved
]

[tool.poetry.scripts]
Expand Down