Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.

Already on GitHub? Sign in to your account

馃摉 Minor typos and copy-editing to checks/write.md #2071

Merged
merged 3 commits into from Jul 26, 2022
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
50 changes: 24 additions & 26 deletions checks/write.md
@@ -1,14 +1,14 @@
# Requirements for a check

If you'd like to add a check, make sure it is something that meets the following
criteria and then create a new GitHub Issue to discuss with the team:
If you'd like to add a check, make sure it meets the following criteria and then
create a new GitHub Issue to discuss with the team:

- The scorecard must only be composed of automate-able, objective data. For
example, a project having 10 contributors doesn鈥檛 necessarily mean it鈥檚 more
secure than a project with say 50 contributors. But, having two maintainers
secure than a project with 50 contributors. But, having two maintainers
might be preferable to only having one - the larger bus factor and ability
to provide code reviews is objectively better.
- The scorecard criteria can be as specific as possible and not limited
- The scorecard criteria can be as specific as possible and are not limited to
general recommendations. For example, for Go, we can recommend/require
specific linters and analyzers to be run on the codebase.
- The scorecard can be populated for any open source project without any work
Expand All @@ -24,60 +24,58 @@ criteria and then create a new GitHub Issue to discuss with the team:

# How to write a check

The steps to writting a check are as follow:
The steps to writing a check are as follows:

1. Create a file under `checks/` folder, say `checks/mycheck.go`
1. Create a file under the `checks/` folder, say `checks/mycheck.go`
2. Give the check a name and register the check:

```
// Note: export the name: start its name with an upper-case letter.
```go
// Note: export the name by starting it with an upper-case letter.
const CheckMyCheckName string = "My-Check"

func init() {
registerCheck(CheckMyCheckName, EntryPointMyCheck)
}
```

3. Log information that is benfical to the user using `checker.DetailLogger`:
3. Log useful information with `checker.DetailLogger`:

* Use `checker.DetailLogger.Warn()` to provide detail on low-score
results. This is showed when the user supplies the `show-results`
option.
results. This is shown when the user supplies the `show-results` option.
* Use `checker.DetailLogger.Info()` to provide detail on high-score
results. This is showed when the user supplies the `show-results`
option.
results. This is shown when the user supplies the `show-results` option.
* Use `checker.DetailLogger.Debug()` to provide detail in verbose mode:
this is showed only when the user supplies the `--verbosity Debug`
this is shown only when the user supplies the `--verbosity Debug`
option.
* If your message relates to a file, try to provide information such as
the `Path`, line number `Offset` and `Snippet`.

4. If the checks fails in a way that is irrecoverable, return a result with
`checker.CreateRuntimeErrorResult()` function: For example, if an error is
returned from an API you call, use the function.
4. If the check fails in a way that is irrecoverable, return a result with the
`checker.CreateRuntimeErrorResult()` function. For example, if an error is
returned from an API you call, use this function.

5. Create the result of the check as follow:
5. Create the result of the check as follows:

* Always provide a high-level sentence explaining the result/score of the
check.
* Always provide a high-level sentence explaining the check's
result/score.
* If the check runs properly but is unable to determine a score, use
`checker.CreateInconclusiveResult()` function.
`checker.CreateInconclusiveResult()`.
* For proportional results, use `checker.CreateProportionalScoreResult()`.
* For maximum score, use `checker.CreateMaxScoreResult()`; for min score
use `checker.CreateMinScoreResult()`.
* For maximum and minimum scores, use `checker.CreateMaxScoreResult()` and
`checker.CreateMinScoreResult()`, respectively.
* If you need more flexibility and need to set a specific score, use
`checker.CreateResultWithScore()` with one of the constants declared,
such as `checker.HalfResultScore`.

6. Dealing with errors: see [errors/errors.md](/errors/errors.md).

7. Create unit tests for both low, high and inconclusive score. Put them in a
file `checks/mycheck_test.go`.
7. Create unit tests for low, high and inconclusive scores. Put them in a file
`checks/mycheck_test.go`.

8. Create e2e tests in `e2e/mycheck_test.go`. Use a dedicated repo that will
not change over time, so that it's reliable for the tests.

9. Update the `checks/checks.yaml` with the description of your check.
9. Update the `checks/checks.yaml` with a description of your check.

10. Generate `docs/check.md` using `make generate-docs`. This will validate and
generate `docs/check.md`.
Expand Down