Enhanced functionality to improve developer experience by identifying…

[dimitris-iliou]

… parent packages or projects from duplicate packages. This commit introduces new features that parse the dependency graph, starting from the duplicate package and ending at the parent package/project that encompasses it.

#6465

[github-advanced-security]

detekt found more than 10 potential problems in the proposed changes. Check the Files changed tab for more details.

[sschuberth]

Thanks or the contribution @dimitris-iliou. But before you put too much effort in it, be informed that during one of the last ORT community meetings it was agreed that the best solution to address the issue is to not have duplicate IDs between project and packages to begin with, and deduplicate these by replacing references with package linkage in the dependency tree by references that link to the project instead.

[codecov]

Codecov Report

Patch and project coverage have no change.

Comparison is base (3fac582) 68.01% compared to head (f2f7c66) 68.01%.

Additional details and impacted files

@@            Coverage Diff            @@
##               main    #7494   +/-   ##
=========================================
  Coverage     68.01%   68.01%           
  Complexity     2023     2023           
=========================================
  Files           339      339           
  Lines         16719    16719           
  Branches       2371     2371           
=========================================
  Hits          11371    11371           
  Misses         4363     4363           
  Partials        985      985           

Flag	Coverage Δ
funTest-docker	69.33% <ø> (ø)
funTest-non-docker	36.46% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[scottschreckengaust]

Thanks or the contribution @dimitris-iliou. But before you put too much effort in it, be informed that during one of the last ORT community meetings it was agreed that the best solution to address the issue is to not have duplicate IDs between project and packages to begin with, and deduplicate these by replacing references with package linkage in the dependency tree by references that link to the project instead.

@sschuberth : How might multiple subdirectories with the same python package dependencies within each requirements.txt file be constructed to not throw an exception?

20:18:16.195 [DefaultDispatcher-worker-1] ERROR java.lang.Object - Multiple projects with the same id 'PIP::src:1b11e6b573ecdb3a109776554dc04462bc00a82b' found. Not adding the project defined in 'https://github.com/organization/repository.git/src/module1/requirements.txt' to the analyzer results as it duplicates the project defined in 'https://github.com/organization/repository/src/module2/requirements.txt'.
Exception in thread "main" java.lang.IllegalArgumentException: Unable to create the AnalyzerResult as it contains packages and projects with the same ids: 
...

from this TODO

ort/analyzer/src/main/kotlin/AnalyzerResultBuilder.kt

Lines 57 to 58 in 956d12e

	// TODO: It might be, e.g. in the case of PIP "requirements.txt" projects, that different projects with
	// the same ID exist. We need to decide how to handle that case.

.

causing this exception

ort/analyzer/src/main/kotlin/AnalyzerResultBuilder.kt

Lines 46 to 47 in 956d12e

	"Unable to create the AnalyzerResult as it contains packages and projects with the same ids: " +
	duplicates.values

For example:

`+ src
|_ src/module1/requirements.txt
|_ src/module2/requirements.txt