feat(coverage): add option to set coverage threshold

[GJZwiers]

This PR implements part of #9669 by adding a --threshold flag to deno coverage that will cause the program to throw an Error if the collected coverage % is lower. Right now there is no default threshold set (well technically 0%) because I am not sure what it should be or if there should even be a default.

The line, branch and function coverages are compared and the program will throw if any one falls below the set threshold.

[bartlomieju]

The total coverage % is calculated by getting the average branch and line coverage and applying some weighting to it (20% branches, 80% lines).

Is there a prior art to that?

[GJZwiers]

The total coverage % is calculated by getting the average branch and line coverage and applying some weighting to it (20% branches, 80% lines).

Is there a prior art to that?

Nope, I was trying to simulate how Coveralls calculates this but I got it wrong. What they actually use is:
$aggregate\ coverage = (lines\ hit + branches\ hit) / (relevant\ lines + relevant\ branches)$
(source)

Alternatively, nyc/istanbul calculate the individual coverages for lines/branches/functions/statements and then if any of those fall under a threshold it throws. So I guess we could go either way.

[bartlomieju]

The total coverage % is calculated by getting the average branch and line coverage and applying some weighting to it (20% branches, 80% lines).

Is there a prior art to that?

Nope, I was trying to simulate how Coveralls calculates this but I got it wrong. What they actually use is: aggregate coverage=(lines hit+branches hit)/(relevant lines+relevant branches) (source)

Alternatively, nyc/istanbul calculate the individual coverages for lines/branches/functions/statements and then if any of those fall under a threshold it throws. So I guess we could go either way.

IMO following nyc/istanbul will be least surprising here.

[aapoalas]

@GJZwiers Will you be working on this to align with nyc/istanbul?

[GJZwiers]

@aapoalas I have not had time to work on this, I'm hoping to have some more time in ~1-1.5 months

[aapoalas]

Hmm. I probably messed up in the merge here and removed the borrow from &script_coverages.

[GJZwiers]

I think I have a spare borrow laying around

[aapoalas]

Looks good to me! Thank you for a magnificent feature!

The code seems good, tests are done, calculation of coverage is implemented according to istanbul which was recommended by @bartlomieju so I see no reason to keep this as a Draft anymore. I'd be willing to merge this as well.

[GJZwiers]

Thanks for approving @aapoalas! I am OK with landing this as is, although I had a few concerns left which is why I kept it as a draft. The current implementation works but is also fairly basic, other implementations have separate flags for each type of coverage e.g. --branches while I used a more general --threshold flag. I can image Deno may want to improve on that in the future, also by somehow collecting statement coverage and including it. Furthermore, I found some comments in an older PR that stated the approach was too tighly coupled to the reporter: #9826 (review), and I think I took more-or-less the same approach. It was two years ago though, so perhaps ideas on that have changed a bit.

[lowlighter]

Any reason for why this was closed without being merged ? It looks close to be shippable

[GJZwiers]

@lowlighter I closed this because I decided to do other things, but the code was pretty much ready to be merged. If you want to make a PR and revive this you are welcome to do so.