Building projects: missing/altered remote dependendencies

[esbena]

@agigleux raised the concern:

dependencies are not longer reachable and so it's no longer possible to compile this Java project that was having a vuln in 2015.
(#67 (comment))

I agree, and note that this also can be a problem for non-compiled languages: it may for instance be relevant to trace flow through an unavailable node_modules dependency.

These are my initial thoughts on how to address the problem, they are ordered by growing complexity.

Solution 1: Record that remote dependencies are missing

A low-tech solution it to add another field to the benchmark entries that indicates that some required remote dependencies are missing for a successful build process:

{
    "CVE": "CVE-YYYY-XXXX",
    "missing-remote-dependencies": true,
    ...
}

A benchmark selector could then filter away such benchmark entries from an evalution like this (a bit verbose):

$ bin/cli list ext:java year:2019 | bin/cli filter --invert-match missing-remote-dependencies > my-buildable-java-cves.txt

Perhaps such benchmark entries should be opt-in instead of opt-out as that would prevent accidental comparison of build-less analysis tools (grep and friends) and build-required analysis tools.

Solution 2: Cache specific dependency kinds

I suppose that most of the disappearing dependencies will be found through the big package managers, and that we could add support for them on a by-need basis.

A simple idea is that we could preemptively upload the dependencies for a build to some global cache that is indexed by the commits of the CVEs.

Drivers could then use that global cache to pre-populate the local cache of machines that are attempting to build the projects at a later date.
I am hypothesising that such a populated local cache could prevent the package managers from attempting to download the missing dependencies.

I note that the operating system or architecture that the builds occur on may also require different dependencies, so the cache may be incomplete.

Solution 3: Cache the internet

Ideally, this project maintained a caching proxy server for all builds such that they did not rely on the ever-changing internet.

I think this is far beyond what this project has resources to do.