Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Basic Attention attribution #148

Merged
merged 35 commits into from Jan 16, 2023
Merged

Basic Attention attribution #148

merged 35 commits into from Jan 16, 2023

Commits on Sep 23, 2022

  1. added jupyterlab dependency (for easier testing)

    @lsickert
    lsickert committed Sep 23, 2022
    Copy the full SHA
    b9ccbf2 View commit details
    Browse the repository at this point in the history

Commits on Oct 17, 2022

  1. initial commit attention methods, added output_attentions parameter t… 

    …o model initialization
    @lsickert
    lsickert committed Oct 17, 2022
    Copy the full SHA
    08d5dbd View commit details
    Browse the repository at this point in the history

Commits on Oct 19, 2022

  1. Merge branch 'main' into attention-attribution

    @lsickert
    lsickert committed Oct 19, 2022
    Copy the full SHA
    57ae54a View commit details
    Browse the repository at this point in the history

Commits on Oct 24, 2022

  1. added basic attention method stubs\n added attention method registry

    @lsickert
    lsickert committed Oct 24, 2022
    Copy the full SHA
    544321c View commit details
    Browse the repository at this point in the history

Commits on Nov 21, 2022

  1. reverted changes to output generation (forward pass done inside attri… 

    …bution function now)
    @lsickert
    lsickert committed Nov 21, 2022
    Copy the full SHA
    d0e859f View commit details
    Browse the repository at this point in the history

  2. first working version of basic attention methods

    @lsickert
    lsickert committed Nov 21, 2022
    Copy the full SHA
    a2a2021 View commit details
    Browse the repository at this point in the history

Commits on Nov 22, 2022

  1. fixed rounding of values in cli output

    @lsickert
    lsickert committed Nov 22, 2022
    Copy the full SHA
    7b3c4fd View commit details
    Browse the repository at this point in the history

  2. added documentation to most methods and generalized functions

    @lsickert
    lsickert committed Nov 22, 2022
    Copy the full SHA
    13fc9f3 View commit details
    Browse the repository at this point in the history

Commits on Nov 23, 2022

  1. Merge branch 'main' into attention-attribution\n\nNeeded to downgrade… 

    … torch to ~1.12.1 again because of platform issue: pytorch/pytorch#88826
    @lsickert
    lsickert committed Nov 23, 2022
    Copy the full SHA
    bb13036 View commit details
    Browse the repository at this point in the history

Commits on Nov 24, 2022

  1. removed python 3.11 build target

    @lsickert
    lsickert committed Nov 24, 2022
    Copy the full SHA
    9340343 View commit details
    Browse the repository at this point in the history

  2. fix safety warnings

    @lsickert
    lsickert committed Nov 24, 2022
    Copy the full SHA
    3cfd706 View commit details
    Browse the repository at this point in the history

Commits on Nov 25, 2022

  1. set correct python version in pyproject.toml

    @lsickert
    lsickert committed Nov 25, 2022
    Copy the full SHA
    2765d63 View commit details
    Browse the repository at this point in the history

  2. regenerated requirements without 3.11

    @lsickert
    lsickert committed Nov 25, 2022
    Copy the full SHA
    4dd442f View commit details
    Browse the repository at this point in the history

Commits on Dec 9, 2022

  1. Merge branch 'main' into attention-attribution, quick fix for mps issue

    @lsickert
    lsickert committed Dec 9, 2022
    Copy the full SHA
    b14407c View commit details
    Browse the repository at this point in the history

Commits on Dec 12, 2022

  1. Merge branch 'main' into attention-attribution

    @lsickert
    lsickert committed Dec 12, 2022
    Copy the full SHA
    6a72166 View commit details
    Browse the repository at this point in the history

Commits on Jan 2, 2023

  1. merge branch 'main' into attention-attribution

    @lsickert
    lsickert committed Jan 2, 2023
    Copy the full SHA
    6535b09 View commit details
    Browse the repository at this point in the history

  2. update deps after merge

    @lsickert
    lsickert committed Jan 2, 2023
    Copy the full SHA
    624435e View commit details
    Browse the repository at this point in the history

  3. include 3.11 as build target

    @lsickert
    lsickert committed Jan 2, 2023
    Copy the full SHA
    06f89a8 View commit details
    Browse the repository at this point in the history

  4. fix different attribution_step argument formatting

    @lsickert
    lsickert committed Jan 2, 2023
    Copy the full SHA
    7bcbe92 View commit details
    Browse the repository at this point in the history

  5. added basic decoder-only support

    @lsickert
    lsickert committed Jan 2, 2023
    Copy the full SHA
    b2fc73c View commit details
    Browse the repository at this point in the history

Commits on Jan 3, 2023

  1. fixed output error for decoder only models

    @lsickert
    lsickert committed Jan 3, 2023
    Copy the full SHA
    b044b4c View commit details
    Browse the repository at this point in the history

  2. removed unnecessary convergence delta references in attention attribu… 

    …tion call
    @lsickert
    lsickert committed Jan 3, 2023
    Copy the full SHA
    8c344b7 View commit details
    Browse the repository at this point in the history

Commits on Jan 4, 2023

  1. allow negative indices when selecting a specific attention head for a… 

    …ttribution
    @lsickert
    lsickert committed Jan 4, 2023
    Copy the full SHA
    f51cf25 View commit details
    Browse the repository at this point in the history

  2. added missing negation to head checking

    @lsickert
    lsickert committed Jan 4, 2023
    Copy the full SHA
    c6a9e70 View commit details
    Browse the repository at this point in the history

  3. fixed last_layer_attention attribution

    @lsickert
    lsickert committed Jan 4, 2023
    Copy the full SHA
    6c9cfae View commit details
    Browse the repository at this point in the history

Commits on Jan 9, 2023

  1. use custom format_attribute_args function for attention methods

    @lsickert
    lsickert committed Jan 9, 2023
    Copy the full SHA
    b78bcc1 View commit details
    Browse the repository at this point in the history

  2. always use decoder_input_embeds in forward output

    @lsickert
    lsickert committed Jan 9, 2023
    Copy the full SHA
    d27f1c3 View commit details
    Browse the repository at this point in the history

  3. reworked LastLayerAttention to work with any single layer and allow a… 

    … range of layers to be specified for averaging in AggregatedAttention
    @lsickert
    lsickert committed Jan 9, 2023
    Copy the full SHA
    cacaa31 View commit details
    Browse the repository at this point in the history

Commits on Jan 10, 2023

  1. Minor bugfixes and version bumps

    @gsarti
    gsarti committed Jan 10, 2023
    Copy the full SHA
    a8d5264 View commit details
    Browse the repository at this point in the history

  2. Generalized attention attribution

    @gsarti
    gsarti committed Jan 10, 2023
    Copy the full SHA
    966f63c View commit details
    Browse the repository at this point in the history

Commits on Jan 13, 2023

  1. updated documentation and added 'min' aggregation function

    @lsickert
    lsickert committed Jan 13, 2023
    Copy the full SHA
    1301a02 View commit details
    Browse the repository at this point in the history

Commits on Jan 14, 2023

  1. Tests, typing fix, additional checks

    @gsarti
    gsarti committed Jan 14, 2023
    Copy the full SHA
    914ee8f View commit details
    Browse the repository at this point in the history

  2. Fix style

    @gsarti
    gsarti committed Jan 14, 2023
    Copy the full SHA
    7c825ad View commit details
    Browse the repository at this point in the history

Commits on Jan 15, 2023

  1. added tests for attention utils

    @lsickert
    lsickert committed Jan 15, 2023
    Copy the full SHA
    f6f0a64 View commit details
    Browse the repository at this point in the history

Commits on Jan 16, 2023

  1. classmethod -> staticmethod where possible

    @gsarti
    gsarti committed Jan 16, 2023
    Copy the full SHA
    f40f63b View commit details
    Browse the repository at this point in the history