Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fully working test on 33 CVEs dataset #342

Merged
merged 76 commits into from Nov 3, 2022
Merged

Fully working test on 33 CVEs dataset #342

merged 76 commits into from Nov 3, 2022

Commits on Oct 10, 2022

  1. keywords extraction test using spacy related to #331, fixes #329

    @sacca97
    sacca97 committed Oct 10, 2022
    Configuration menu
    Copy the full SHA
    dc06c76 View commit details
    Browse the repository at this point in the history

  2. added spacy en_core_web_sm to python.yml for gh workflows

    @sacca97
    sacca97 committed Oct 10, 2022
    Configuration menu
    Copy the full SHA
    bc66866 View commit details
    Browse the repository at this point in the history

  3. fixed json report generation to handle sets and file extensions when … 

    …default is set
    @sacca97
    sacca97 committed Oct 10, 2022
    Configuration menu
    Copy the full SHA
    a389c11 View commit details
    Browse the repository at this point in the history

  4. flake8 failure

    @sacca97
    sacca97 committed Oct 10, 2022
    Configuration menu
    Copy the full SHA
    ae9ddfb View commit details
    Browse the repository at this point in the history

  5. initial lsh implementation

    @sacca97
    sacca97 committed Oct 10, 2022
    Configuration menu
    Copy the full SHA
    e57af9b View commit details
    Browse the repository at this point in the history

  6. initial lsh implementation

    @sacca97
    sacca97 committed Oct 10, 2022
    Configuration menu
    Copy the full SHA
    516d35c View commit details
    Browse the repository at this point in the history

  7. Merge branch 'twin-commits' of github.com:SAP/project-kb into twin-co… 

    …mmits
    @sacca97
    sacca97 committed Oct 10, 2022
    Configuration menu
    Copy the full SHA
    56a7785 View commit details
    Browse the repository at this point in the history

  8. add datasketch to requirements.txt

    @sacca97
    sacca97 committed Oct 10, 2022
    Configuration menu
    Copy the full SHA
    43cd0f2 View commit details
    Browse the repository at this point in the history

Commits on Oct 12, 2022

  1. rules refactoring, now every rule is a sub-class of the class Rule

    @sacca97
    sacca97 committed Oct 12, 2022
    Configuration menu
    Copy the full SHA
    a71ca7d View commit details
    Browse the repository at this point in the history

  2. finished rules refactoring, added Minhash to Commit: reworked backend… 

    … and abstracted the commits eliminating some intra-dependencies
    @sacca97
    sacca97 committed Oct 12, 2022
    Configuration menu
    Copy the full SHA
    e7a55ac View commit details
    Browse the repository at this point in the history

Commits on Oct 13, 2022

  1. code cleanup, report code refactoring

    @sacca97
    sacca97 committed Oct 13, 2022
    Configuration menu
    Copy the full SHA
    a2e315e View commit details
    Browse the repository at this point in the history

  2. added find_twin method, testing phase

    @sacca97
    sacca97 committed Oct 13, 2022
    Configuration menu
    Copy the full SHA
    43ebadb View commit details
    Browse the repository at this point in the history

  3. minor addons and cleanup

    @sacca97
    sacca97 committed Oct 13, 2022
    Configuration menu
    Copy the full SHA
    9a58018 View commit details
    Browse the repository at this point in the history

  4. Major refactoring, cleanup and optimization. WIP

    @sacca97
    sacca97 committed Oct 13, 2022
    Configuration menu
    Copy the full SHA
    64ce70a View commit details
    Browse the repository at this point in the history

  5. continuing refactoring and fixes

    @sacca97
    sacca97 committed Oct 13, 2022
    Configuration menu
    Copy the full SHA
    c93db29 View commit details
    Browse the repository at this point in the history

Commits on Oct 14, 2022

  1. almost finished Git, raw_commit and exec refactoring and cleaning

    @sacca97
    sacca97 committed Oct 14, 2022
    Configuration menu
    Copy the full SHA
    efbc0e7 View commit details
    Browse the repository at this point in the history

  2. other refactoring, fixed logging level and refactored it

    @sacca97
    sacca97 committed Oct 14, 2022
    Configuration menu
    Copy the full SHA
    f3e4906 View commit details
    Browse the repository at this point in the history

  3. old files cleanup

    @sacca97
    sacca97 committed Oct 14, 2022
    Configuration menu
    Copy the full SHA
    72ae9f9 View commit details
    Browse the repository at this point in the history

Commits on Oct 17, 2022

  1. fixed typos, updated pytest files, modified db structure and updated … 

    …python postgres class to dynamically create the insert statement based on the data
    @sacca97
    sacca97 committed Oct 17, 2022
    Configuration menu
    Copy the full SHA
    f46304e View commit details
    Browse the repository at this point in the history

  2. modified logger class to wrap the default python logger

    @sacca97
    sacca97 committed Oct 17, 2022
    Configuration menu
    Copy the full SHA
    93dd2bc View commit details
    Browse the repository at this point in the history

  3. fixed logger, updated some default parameters

    @sacca97
    sacca97 committed Oct 17, 2022
    Configuration menu
    Copy the full SHA
    6d4fa1c View commit details
    Browse the repository at this point in the history

  4. adjusted some imports to match new refactoring, adjusted some types

    @sacca97
    sacca97 committed Oct 17, 2022
    Configuration menu
    Copy the full SHA
    46df41a View commit details
    Browse the repository at this point in the history

  5. re-fixed logger

    @sacca97
    sacca97 committed Oct 17, 2022
    Configuration menu
    Copy the full SHA
    4103d09 View commit details
    Browse the repository at this point in the history

  6. updated method names and tests, added verb extraction from text

    @sacca97
    sacca97 committed Oct 17, 2022
    Configuration menu
    Copy the full SHA
    d44ef50 View commit details
    Browse the repository at this point in the history

  7. testing small relevant extensions version

    @sacca97
    sacca97 committed Oct 17, 2022
    Configuration menu
    Copy the full SHA
    09ebb94 View commit details
    Browse the repository at this point in the history

  8. added automatic filtering for different file extension at git level w… 

    …ithout spacifying with the cli
    @sacca97
    sacca97 committed Oct 17, 2022
    Configuration menu
    Copy the full SHA
    157879f View commit details
    Browse the repository at this point in the history

  9. minor fixes to raw commit class

    @sacca97
    sacca97 committed Oct 17, 2022
    Configuration menu
    Copy the full SHA
    56f4890 View commit details
    Browse the repository at this point in the history

Commits on Oct 18, 2022

  1. finished git rework, added setup.py to allow external imports

    @sacca97
    sacca97 committed Oct 18, 2022
    Configuration menu
    Copy the full SHA
    84b9952 View commit details
    Browse the repository at this point in the history

  2. updated setup.py

    @sacca97
    sacca97 committed Oct 18, 2022
    Configuration menu
    Copy the full SHA
    17c1ff4 View commit details
    Browse the repository at this point in the history

Commits on Oct 20, 2022

  1. updated makefile and .gitignore

    @sacca97
    sacca97 committed Oct 20, 2022
    Configuration menu
    Copy the full SHA
    ce29db8 View commit details
    Browse the repository at this point in the history

  2. modified and updated logger

    @sacca97
    sacca97 committed Oct 20, 2022
    Configuration menu
    Copy the full SHA
    39e55a0 View commit details
    Browse the repository at this point in the history

  3. reworked RawCommit and Git to reduce the calls to subprocess, overall… 

    … speed improvement in preprocessing.
    @sacca97
    sacca97 committed Oct 20, 2022
    Configuration menu
    Copy the full SHA
    62cf3a3 View commit details
    Browse the repository at this point in the history

  4. minor changes

    @sacca97
    sacca97 committed Oct 20, 2022
    Configuration menu
    Copy the full SHA
    f95b166 View commit details
    Browse the repository at this point in the history

  5. console fixing tests

    @sacca97
    sacca97 committed Oct 20, 2022
    Configuration menu
    Copy the full SHA
    1ed5a56 View commit details
    Browse the repository at this point in the history

  6. fixed report json export

    @sacca97
    sacca97 committed Oct 20, 2022
    Configuration menu
    Copy the full SHA
    46a1c31 View commit details
    Browse the repository at this point in the history

  7. added lemmatization to spacy, optimized and cleaned jira and github r… 

    …eferences extraction
    @sacca97
    sacca97 committed Oct 20, 2022
    Configuration menu
    Copy the full SHA
    61e6f11 View commit details
    Browse the repository at this point in the history

  8. updated helper method to match nlp extraction format

    @sacca97
    sacca97 committed Oct 20, 2022
    Configuration menu
    Copy the full SHA
    03959e2 View commit details
    Browse the repository at this point in the history

  9. fixed a "black hole" bug that was causing some commits to disappear

    @sacca97
    sacca97 committed Oct 20, 2022
    Configuration menu
    Copy the full SHA
    15aae0c View commit details
    Browse the repository at this point in the history

  10. fixed imports and modified names to match real usage

    @sacca97
    sacca97 committed Oct 20, 2022
    Configuration menu
    Copy the full SHA
    0013566 View commit details
    Browse the repository at this point in the history

  11. fixed imports and modified the names to match real usage

    @sacca97
    sacca97 committed Oct 20, 2022
    Configuration menu
    Copy the full SHA
    9f3957f View commit details
    Browse the repository at this point in the history

  12. web scraping to match nlp extraction

    @sacca97
    sacca97 committed Oct 20, 2022
    Configuration menu
    Copy the full SHA
    9261ac6 View commit details
    Browse the repository at this point in the history

Commits on Oct 22, 2022

  1. updated gitignore

    @sacca97
    sacca97 committed Oct 22, 2022
    Configuration menu
    Copy the full SHA
    e6c2a17 View commit details
    Browse the repository at this point in the history

Commits on Oct 24, 2022

  1. deleted some useless stuff, renamed test files

    @sacca97
    sacca97 committed Oct 24, 2022
    Configuration menu
    Copy the full SHA
    0c49bdb View commit details
    Browse the repository at this point in the history

  2. updated test filenames. TODO: Fix stuff

    @sacca97
    sacca97 committed Oct 24, 2022
    Configuration menu
    Copy the full SHA
    fe48ee8 View commit details
    Browse the repository at this point in the history

Commits on Oct 25, 2022

  1. updated lsh helpers method

    @sacca97
    sacca97 committed Oct 25, 2022
    Configuration menu
    Copy the full SHA
    a467aef View commit details
    Browse the repository at this point in the history

  2. back to basemodel usage

    @sacca97
    sacca97 committed Oct 25, 2022
    Configuration menu
    Copy the full SHA
    517e294 View commit details
    Browse the repository at this point in the history

  3. added xlm parser for jira issues

    @sacca97
    sacca97 committed Oct 25, 2022
    Configuration menu
    Copy the full SHA
    1f05996 View commit details
    Browse the repository at this point in the history

  4. updated rules with twins

    @sacca97
    sacca97 committed Oct 25, 2022
    Configuration menu
    Copy the full SHA
    cd945e1 View commit details
    Browse the repository at this point in the history

  5. report handlers now create also the nested directories

    @sacca97
    sacca97 committed Oct 25, 2022
    Configuration menu
    Copy the full SHA
    e559ca7 View commit details
    Browse the repository at this point in the history

  6. refactoring. now handling large requests to the preprocessed_commits … 

    …backend properly (only get, not yet post)
    @sacca97
    sacca97 committed Oct 25, 2022
    Configuration menu
    Copy the full SHA
    6bf338e View commit details
    Browse the repository at this point in the history

  7. minor updates and optimizations

    @sacca97
    sacca97 committed Oct 25, 2022
    Configuration menu
    Copy the full SHA
    e53333a View commit details
    Browse the repository at this point in the history

  8. finished refactoring and optimization only calling "git log" once for… 

    … all the commits
    @sacca97
    sacca97 committed Oct 25, 2022
    Configuration menu
    Copy the full SHA
    5b285c0 View commit details
    Browse the repository at this point in the history

  9. minor updates to match other changes

    @sacca97
    sacca97 committed Oct 25, 2022
    Configuration menu
    Copy the full SHA
    b25084a View commit details
    Browse the repository at this point in the history

  10. updated database structure, again

    @sacca97
    sacca97 committed Oct 25, 2022
    Configuration menu
    Copy the full SHA
    5045e21 View commit details
    Browse the repository at this point in the history

  11. update to match get_hunks

    @sacca97
    sacca97 committed Oct 25, 2022
    Configuration menu
    Copy the full SHA
    f9c7eb6 View commit details
    Browse the repository at this point in the history

Commits on Nov 2, 2022

  1. update to python 3.10

    @sacca97
    sacca97 committed Nov 2, 2022
    Configuration menu
    Copy the full SHA
    bc32ed4 View commit details
    Browse the repository at this point in the history

  2. updated html report format

    @sacca97
    sacca97 committed Nov 2, 2022
    Configuration menu
    Copy the full SHA
    42ed7b9 View commit details
    Browse the repository at this point in the history

  3. now jira issues are accessed using XML API

    @sacca97
    sacca97 committed Nov 2, 2022
    Configuration menu
    Copy the full SHA
    12eb250 View commit details
    Browse the repository at this point in the history

  4. updated rule messages and fully functioning twin commits

    @sacca97
    sacca97 committed Nov 2, 2022
    Configuration menu
    Copy the full SHA
    f8bbba4 View commit details
    Browse the repository at this point in the history

  5. reworked communication with backend

    @sacca97
    sacca97 committed Nov 2, 2022
    Configuration menu
    Copy the full SHA
    528d4d0 View commit details
    Browse the repository at this point in the history

  6. fixed bug in retrieving preprocessed commits

    @sacca97
    sacca97 committed Nov 2, 2022
    Configuration menu
    Copy the full SHA
    db23d0c View commit details
    Browse the repository at this point in the history

  7. skip github when fetching references, we already do it separately

    @sacca97
    sacca97 committed Nov 2, 2022
    Configuration menu
    Copy the full SHA
    48168b2 View commit details
    Browse the repository at this point in the history

  8. added github apis, fine-tuned word extraction

    @sacca97
    sacca97 committed Nov 2, 2022
    Configuration menu
    Copy the full SHA
    c04e9db View commit details
    Browse the repository at this point in the history

  9. allow single-hunk commits

    @sacca97
    sacca97 committed Nov 2, 2022
    Configuration menu
    Copy the full SHA
    5d0a87c View commit details
    Browse the repository at this point in the history

  10. code cleanup

    @sacca97
    sacca97 committed Nov 2, 2022
    Configuration menu
    Copy the full SHA
    3764a31 View commit details
    Browse the repository at this point in the history

  11. fine-tuned security words extraction

    @sacca97
    sacca97 committed Nov 2, 2022
    Configuration menu
    Copy the full SHA
    d2d6a89 View commit details
    Browse the repository at this point in the history

  12. kinda-final lsh implementation

    @sacca97
    sacca97 committed Nov 2, 2022
    Configuration menu
    Copy the full SHA
    9f6c261 View commit details
    Browse the repository at this point in the history

  13. back to full basemodel

    @sacca97
    sacca97 committed Nov 2, 2022
    Configuration menu
    Copy the full SHA
    b2bd8d8 View commit details
    Browse the repository at this point in the history

Commits on Nov 3, 2022

  1. adapted tests

    @sacca97
    sacca97 committed Nov 3, 2022
    Configuration menu
    Copy the full SHA
    4170ca8 View commit details
    Browse the repository at this point in the history

  2. minor changes

    @sacca97
    sacca97 committed Nov 3, 2022
    Configuration menu
    Copy the full SHA
    4623541 View commit details
    Browse the repository at this point in the history

  3. Merge branch 'main' into twin-commits

    @sacca97
    sacca97 committed Nov 3, 2022
    Configuration menu
    Copy the full SHA
    ff056ae View commit details
    Browse the repository at this point in the history

  4. changed gh workflows for python 3.10

    @sacca97
    sacca97 committed Nov 3, 2022
    Configuration menu
    Copy the full SHA
    2bae2bc View commit details
    Browse the repository at this point in the history

  5. Merge branch 'twin-commits' of github.com:SAP/project-kb into twin-co… 

    …mmits
    @sacca97
    sacca97 committed Nov 3, 2022
    Configuration menu
    Copy the full SHA
    1d1d1d9 View commit details
    Browse the repository at this point in the history

  6. Merge branch 'twin-commits' of github.com:SAP/project-kb into twin-co… 

    …mmits
    @sacca97
    sacca97 committed Nov 3, 2022
    Configuration menu
    Copy the full SHA
    f2c8c2c View commit details
    Browse the repository at this point in the history

  7. updated github actions for python

    @sacca97
    sacca97 committed Nov 3, 2022
    Configuration menu
    Copy the full SHA
    0cbde17 View commit details
    Browse the repository at this point in the history

  8. updated github actions for python

    @sacca97
    sacca97 committed Nov 3, 2022
    Configuration menu
    Copy the full SHA
    ee7c1ad View commit details
    Browse the repository at this point in the history