Clarify rules for curations and package configurations

[tsteenbe]

Propose to align on set of common rules for curations and package configurations to make it clear and concise for any contributors what is expected and why. Having rules will also help guide the auto generation of these files using orthw.

Below I took a first stab at proposing some rules - I invite all to propose their rules as input for an upcoming ORT TSC discussion.

General

Rule: Re-use the same wording for each type of curations and package configuration
Motivation:
To facilitate automation and ability to understand entries we align on same wording in comments across the ort-config repository each type of curations and package configuration. An example of this would be to use the same sentence for say each VCS path or declared license mapping.

Rule: Use >- over | for YAML multiline strings
Motivation:
Using a | preserves newlines in YAML but it results in visible \n in the resolved configuration in an ORT analyzer result. Using >- which strips new lines and folds lines does not have this issue

Rule: Links should be immutable
Motivation:
Links in comments are used to provide proof to support the curations or package configurations. If a links is not immutable by pointing to say main, master or develop branch of Git repository which likely changes overtime then verifying the proof becomes difficult.

Curations

Rule: Each curation must come with proof
Motivation:
The reason why the curation is needed/valid must be clear to all.

Rule: Individual curations should do only 1 thing
Motivation:
Mixing multiple correction says a VCS url and path curation and a concluded license curations in one block makes review harder and prevents future automation.

Rule: Namespace curations are not allowed
Motivation:
Curations that apply to whole namespaces by only setting the type and namespace of the identifier are not allowed as risk of overwriting valid data due to overbroad package id matching is too big.

Rule: Declared license mapping curations are made without specifying version (range) unless there evidence or relicensing
Motivation:
It’s generally safe to apply a mapping across any version as if a mapping does not map anymore the effect is zero. There is a risk that say a Python library which is ‘BSD licensed’ => BSD-2-Clause becomes invalid when a relicensing to BSD-3-Clause happens. However, from our experience this type of within the same family relicensing is extremely rare. If the package has been relicensed then version ranges should be used to denote the range for which the mapping is valid

Rule: VCS path curations must be for a specific version?
Motivation:
???

Package Configurations

Rule: Path excludes should use as broad matching patterns as much as possible
Motivation:
The larger the amount of path excludes the slower ORT’s evaluator/reporter execution gets so use ‘/tests/’ instead of specifying tens of absolute directory paths to various tests directories.

Rule: Each license finding curation concluded license must come with proof
Motivation:
The reason why the curation is needed/valid must be clear to all.

Rule: Concluded license in license finding curations must preserve license choices
Motivation:
License choices are project/organization-specific and it not upon the ORT community to evaluate which license is better one in case of choice between multiple.

[sschuberth]

On top of any written contribution rules, we should add automated checks for them wherever possible, as experience shows that contribution guidelines are barely read by anyone.

Actually, I'd even favor automated checks with clear failure messages (that are posted as PR comments instead of being hidden in CI logs) over having basically the same texts written in some contribution guidelines for rules.

[tsteenbe]

Two more rules clarifications:

Curations

Rule: Use pre-defined commit message when adding curations.
Motivation:
Enables clear guidance for contributors, can be automated reducing reviewers time and consistent message makes changes easier to understand.

Examples:

• feat(curations): Conclude license for <package id>
• feat(curations): Set VCS path for <package id>
• feat(curations): Map declared license for <package id>

Package configurations

Rule: Concluded license curation must come with scan results and ScanCode version in commit message
Motivation:
Enable pull request reviewers to verify correctness of the curation and ScanCode version so it possible to reproduce the scan in the future.

Rule: Use pre-defined commit message when adding package configurations.
Motivation:
Enables clear guidance for contributors, can be automated reducing reviewers time and consistent message makes changes easier to understand.

Examples:

• feat(package-configs): Add excludes for <package id>
• feat(package-configs): Add curations for <package id>