MVP script to sync github advisory data in #392

rschultheis · 2019-06-14T18:17:56Z

This is a script which imports data from the GitHub Security Advisory API into this advisory database.

Basic overview

Person runs the script using: bundle exec rake sync_github_advisories
Script writes yaml advisory files for all rubygem advisories that GitHub has, which RubySec does not already have
Person reviews each file carefully, makes sure data looks right
Person has to set the patched_versions and unaffected_versions versions manually, using comments in the file. GitHub data is structured opposite of RubySec unfortunately: GitHub identifies version range which are vulnerable, RubySec identifies version ranges which are not vulnerable. The translation from the GitHub way to the RubySec way is manual (but assisted by comments made by this script in the new yaml files it makes).
Person decides which of those files to make PRs for and makes those PRs manually.

Running instructions

The GitHub Advisory API requires a token to access it.

It can be a completely scopeless token (recommended), it does not require any permissions at all.
Get yours at https://github.com/settings/tokens

export GH_API_TOKEN=<your GitHub API Token>
bundle exec rake sync_github_advisories

Examples

So far I've made two PRs using this script:

There are a handful more recent CVEs that we could sync over now. There are many for older years that are simply not as interesting, and potentially not worth the energy of ingesting. I am open to feedback on this.

rschultheis · 2019-06-18T18:21:06Z

I am opening this up for review! There are now two example PRs made by this script so I think that is good enough to get the idea across. Is anyone on the rubysec squad interested in this work?

reedloden · 2019-06-26T09:01:02Z

@rschultheis sorry for the delay!

This looks fantastic. So thankful for all the hard work here. Very interested in this.

reedloden · 2019-06-26T09:01:49Z

lib/github_advisory_sync.rb

+          "cve" => cve_id[4..20],
+          "date" => published_day,
+          "url" => external_reference,
+          "title" => github_advisory_graphql_object["summary"],


Is there any data that would allow us to have a better title for the issues besides just "SEVERITY_LEVEL severity vulnerability that affects GEM"? Like is there any CWE or similar data that could be used to customize the generated titles?

well, another limitation of how the GitHub api currently works. The summary field is a bit aspirational in the GitHub Advisory object right now, and it is literally that template you define. For CVEs there is no "summary" field in CVEs, just description, so only way to do this is to write a better one. Or take the first sentence of the CVE description perhaps.

reedloden · 2019-06-26T09:02:25Z

lib/github_advisory_sync.rb

+    end
+
+    def external_reference
+      github_advisory_graphql_object["references"].first["url"]


Is this always NVD? Wish we could use something else... Would prefer the url not always just be a link back to NVD.

So, the GitHub API currently only provides a single reference unfortunately. Its a big limitation in this case for sure. For CVEs the GitHub primary reference is the nvd link. When GitHub ultimately does publish a maintainer security advisory for a rubygem, the link though will be to the GitHub advisory(NPM example).

We could hand curate in some better references for sure. Option B would be to grab the CVE data and pull the reference list from that. It seems worth updating the docs at least to mention updating the references to include best/most important references not just what was in the GitHub API.

reedloden · 2019-06-26T09:03:48Z

lib/github_advisory_sync.rb

+          # and therefore should not be committed as is
+          # We can not directly translate from GitHub to rubysec advisory format
+          #
+          # The patched_versions field is not exactly available.


Can you provide some examples of what you see here? I wonder if we could guess a bit, knowing that it still needs human review but might make the process easier?

so it is definitely possible to make some logic to automate the most common cases of translation.

Some common cases:

{ "package": { "name": "chloride" }, "vulnerableVersionRange": "< 0.3.0", "firstPatchedVersion": { "identifier": "0.3.0" } },

{ "package": { "name": "actionview" }, "vulnerableVersionRange": ">= 4.0.0, < 4.2.11.1", "firstPatchedVersion": { "identifier": "4.2.11.1" } }, { "package": { "name": "actionview" }, "vulnerableVersionRange": ">= 5.2.0, < 5.2.2.1", "firstPatchedVersion": { "identifier": "5.2.2.1" } },

{ "package": { "name": "actionview" }, "vulnerableVersionRange": ">= 4.0.0, < 4.2.11.1", "firstPatchedVersion": { "identifier": "4.2.11.1" } }, { "package": { "name": "actionview" }, "vulnerableVersionRange": ">= 5.2.0, < 5.2.2.1", "firstPatchedVersion": { "identifier": "5.2.2.1" } },

I think my preference would be to follow this logic up with another PR. We might need to add some tests for the smart logic like that and this isn't really structured right now for that

reedloden · 2019-06-26T09:05:24Z

lib/github_advisory_sync.rb

+    # There are many old CVEs in the GitHub advisory dataset that are not in here
+    # It is more important to sync the newer ones, so this allows the user to
+    # control how old of CVEs the sync should pull over
+    def self.sync(min_year: 2018)


How many does the script output for 2018 onward?

For 2018 onward, it produces this output:

Sync completed Wrote these files: --- - gems/rubygems-update/CVE-2019-8325.yml - gems/rubygems-update/CVE-2019-8320.yml - gems/rubygems-update/CVE-2019-8324.yml - gems/rubygems-update/CVE-2019-8321.yml - gems/rubygems-update/CVE-2019-8322.yml - gems/rubygems-update/CVE-2019-8323.yml - gems/ruby-openid/CVE-2019-11027.yml - gems/chloride/CVE-2018-6517.yml - gems/doorkeeper-openid_connect/CVE-2019-9837.yml - gems/fat_free_crm/CVE-2018-1000842.yml - gems/smart_proxy_dynflow/CVE-2018-14643.yml - gems/jekyll/CVE-2018-17567.yml - gems/bootstrap/CVE-2018-14042.yml

The six for rubygems are actually already in this repo over at https://github.com/rubysec/ruby-advisory-db/tree/master/libraries/rubygems

so it is syncing over 13 CVEs, but only 7 are legit additions I think.

If we look at at all time, the stats work out this:

8 2011 11 2012 15 2013 11 2014 1 2015 1 2017 5 2018 8 2019

All that is simply CVEs, which is all that is currently published by GitHub. The real gain is the future...

rschultheis

Hi @reedloden thanks for maintaining RubySec.

So full disclosure I work at GitHub on the Advisory dataset (with @mveytsman and @phillmv 👋 ).

So right now the only rubygem advisories in GitHub's api are CVE based. But the new maintainer security advisories should change that soon. With that feature a gem maintainer can report an advisory to GitHub, and if it meets GitHub's requirements it gets published, and then this script will sync it down. So in this way we should get advisories which are not CVE based, but rather straight from gem maintainers. Here is an example NPM advisory which has been published in this way.

Let me know if you think any other changes should happen and if you are cool with merging this!

rschultheis · 2019-06-28T02:19:21Z

lib/github_advisory_sync.rb

+    # There are many old CVEs in the GitHub advisory dataset that are not in here
+    # It is more important to sync the newer ones, so this allows the user to
+    # control how old of CVEs the sync should pull over
+    def self.sync(min_year: 2018)


For 2018 onward, it produces this output:

Sync completed Wrote these files: --- - gems/rubygems-update/CVE-2019-8325.yml - gems/rubygems-update/CVE-2019-8320.yml - gems/rubygems-update/CVE-2019-8324.yml - gems/rubygems-update/CVE-2019-8321.yml - gems/rubygems-update/CVE-2019-8322.yml - gems/rubygems-update/CVE-2019-8323.yml - gems/ruby-openid/CVE-2019-11027.yml - gems/chloride/CVE-2018-6517.yml - gems/doorkeeper-openid_connect/CVE-2019-9837.yml - gems/fat_free_crm/CVE-2018-1000842.yml - gems/smart_proxy_dynflow/CVE-2018-14643.yml - gems/jekyll/CVE-2018-17567.yml - gems/bootstrap/CVE-2018-14042.yml

The six for rubygems are actually already in this repo over at https://github.com/rubysec/ruby-advisory-db/tree/master/libraries/rubygems

so it is syncing over 13 CVEs, but only 7 are legit additions I think.

If we look at at all time, the stats work out this:

8 2011 11 2012 15 2013 11 2014 1 2015 1 2017 5 2018 8 2019

All that is simply CVEs, which is all that is currently published by GitHub. The real gain is the future...

rschultheis · 2019-06-28T02:25:57Z

lib/github_advisory_sync.rb

+    end
+
+    def external_reference
+      github_advisory_graphql_object["references"].first["url"]


So, the GitHub API currently only provides a single reference unfortunately. Its a big limitation in this case for sure. For CVEs the GitHub primary reference is the nvd link. When GitHub ultimately does publish a maintainer security advisory for a rubygem, the link though will be to the GitHub advisory(NPM example).

We could hand curate in some better references for sure. Option B would be to grab the CVE data and pull the reference list from that. It seems worth updating the docs at least to mention updating the references to include best/most important references not just what was in the GitHub API.

rschultheis · 2019-06-28T02:27:46Z

lib/github_advisory_sync.rb

+          "cve" => cve_id[4..20],
+          "date" => published_day,
+          "url" => external_reference,
+          "title" => github_advisory_graphql_object["summary"],


well, another limitation of how the GitHub api currently works. The summary field is a bit aspirational in the GitHub Advisory object right now, and it is literally that template you define. For CVEs there is no "summary" field in CVEs, just description, so only way to do this is to write a better one. Or take the first sentence of the CVE description perhaps.

rschultheis · 2019-06-28T02:34:11Z

lib/github_advisory_sync.rb

+          # and therefore should not be committed as is
+          # We can not directly translate from GitHub to rubysec advisory format
+          #
+          # The patched_versions field is not exactly available.


so it is definitely possible to make some logic to automate the most common cases of translation.

Some common cases:

{ "package": { "name": "chloride" }, "vulnerableVersionRange": "< 0.3.0", "firstPatchedVersion": { "identifier": "0.3.0" } },

{ "package": { "name": "actionview" }, "vulnerableVersionRange": ">= 4.0.0, < 4.2.11.1", "firstPatchedVersion": { "identifier": "4.2.11.1" } }, { "package": { "name": "actionview" }, "vulnerableVersionRange": ">= 5.2.0, < 5.2.2.1", "firstPatchedVersion": { "identifier": "5.2.2.1" } },

{ "package": { "name": "actionview" }, "vulnerableVersionRange": ">= 4.0.0, < 4.2.11.1", "firstPatchedVersion": { "identifier": "4.2.11.1" } }, { "package": { "name": "actionview" }, "vulnerableVersionRange": ">= 5.2.0, < 5.2.2.1", "firstPatchedVersion": { "identifier": "5.2.2.1" } },

I think my preference would be to follow this logic up with another PR. We might need to add some tests for the smart logic like that and this isn't really structured right now for that

reedloden · 2019-06-28T17:58:42Z

Sounds good. Just landed this. Let's iterate and figure out next steps.

MVP script to sync github advisory data in

7c00292

rschultheis mentioned this pull request Jun 18, 2019

Add CVE-2019-9837 for doorkeeper-openid_connect #393

Merged

github sync script updates

b9cb562

rschultheis mentioned this pull request Jun 18, 2019

add CVE-2018-17567 on jekyll #394

Merged

rschultheis marked this pull request as ready for review June 18, 2019 18:19

rschultheis added 2 commits June 19, 2019 08:28

minor improvements to github sync script

6a0e1fd

Readme explains how to run GitHub Advisory sync

97a727e

reedloden reviewed Jun 26, 2019

View reviewed changes

rschultheis commented Jun 28, 2019

View reviewed changes

reedloden merged commit 6744361 into rubysec:master Jun 28, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

MVP script to sync github advisory data in #392

MVP script to sync github advisory data in #392

rschultheis commented Jun 14, 2019 •

edited

rschultheis commented Jun 18, 2019

reedloden commented Jun 26, 2019

reedloden Jun 26, 2019

rschultheis Jun 28, 2019

reedloden Jun 26, 2019

rschultheis Jun 28, 2019

reedloden Jun 26, 2019

rschultheis Jun 28, 2019

reedloden Jun 26, 2019

rschultheis Jun 28, 2019

rschultheis left a comment

rschultheis Jun 28, 2019

rschultheis Jun 28, 2019

rschultheis Jun 28, 2019

rschultheis Jun 28, 2019

reedloden commented Jun 28, 2019

MVP script to sync github advisory data in #392

MVP script to sync github advisory data in #392

Conversation

rschultheis commented Jun 14, 2019 • edited

Basic overview

Running instructions

Examples

rschultheis commented Jun 18, 2019

reedloden commented Jun 26, 2019

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rschultheis left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

reedloden commented Jun 28, 2019

rschultheis commented Jun 14, 2019 •

edited