New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add new format abstraction #543
Closed
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
Closing to keep the context here --a new PR will introduce the new pattern and one ported format |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This is not ready for final review yet for a few reasons:
Partially addresses anchore/grype#395
Today we have the
presenter
abstraction to write out internal SBOM results to a particular format (e.g. JSON, table, SPDX, CycloneDX). All model details about the format itself is contained under the presenter objects themselves. This abstraction was never meant to encapsulate handling intake (parsing) of these formats. As we add more formats it would be ideal to be able to parse these formats for use in other tools (such as syft). For this reason this PR adds a newformat
abstraction for specifying encoding/decoding details for a particular SBOM format (and the presenter can use format encoders to deal with presentation concerns).The first commit of this PR adds the new format abstractions:
type Encoder ...
: a function signature for encoding a given set of SBOM objects into bytes that are written to a writer.type Decoder ...
: a function signature for decoding an SBOM from a reader and returning SBOM objects.type Validator ...
: a function signature for observing the bytes of an SBOM document via a reader and returns any errors if the given document is not of a specific format.type Format ...
: ties together the above bits of functionality into a single object tied to a specificformat.Option
(with helper functions)This new pattern allows for:
This PR has moved a lot of code and large test fixtures. To better orient what is really happening I'd suggest reviewing a commit at a time in order... but these are the most important ones:
format
abstraction first: 9ded1c4The remaining commits involve adding tests, moving existing test-fixtures, removing some presenter implementations, and making common SPDX helpers available to other packages.... essentially it's what "shakes out" after getting the abstraction and initial implementations added.
Why move both JSON and SPDX-JSON? I attempted to move just SPDX-JSON to implement anchore/grype#395, however, in doing so there was a lot of overlap with the JSON decoding implementation in grype. Instead of having this live in two places it was easier conceptually to go ahead an move the JSON decoding into the new approach.
Regarding details of the SPDX-JSON fields: new fields were added to the SPDX-JSON structs such that encoding-decoding loops are NOT lossy. That is, Before this PR if you were to encoding a SPDX-JSON document it would be lossy relative to the internal syft document... with the adjustments in this PR this is no longer the case. Why do this? In order to support keeping the
package.Metadata
fields that are sensitive to vulnerability scanning in grype. However, selecting ONLY the fields that grype needs is brittle from syft's point of view ("why were these fields persisted and those not?!?"). The initial implementation is lossless for this reason (and the added reason that selecting a different shape would require custompkg.*Metadata
struct definitions for each customization, which seems difficult to maintain!).