First class relationships #607

wagoodman · 2021-10-31T15:37:55Z

This PR primarily introduces the concept of an artifact.Relationship (instead of a pkg.Relationship) and adds a new element to the sbom.SBOM struct (as a root level element, a sibling of the Artifact field).

As a result the following additions/changes were also made to support introducing artifact.Relationship:

Create the idea of an artifact.ID as a type
Introduce a new Identifiable interface for objects that can return a stable artifact.ID (and thereby be able to be part of a artifact.Relationship)
Hoist up the package fingerprinting to be a general purpose utility in the artifact package
Hoist up the existing ownership-by-file-overlap relationship to now be discovered/created just after the cataloging phase (instead of within the syftjson format encoder as it is today)
Ignore select metadata struct fields and made subtle changes to CPE generation behavior to further stabilize the package ID (based on an object hash)
Migrate the encode-decode cycle test as an integration test (utilizing all of the known package types) to prove that a package ID remains stable upon multiple encode-decode cycles
split source.Location into source.Coordinates. This new object has the minimal information needed to get to a path within a source. This new source.Coodinates implements artifact.Identifiable such that files can form relationships with packages (setting up for Port the SPDX JSON files relationships in a separate PR) This work has been broken out into a separate PR

Additional considerations:

I attempted to remove the pkg.Catalog altogether, however, elected to leave it for now to keep this PR smaller and increment towards a final state in a future TBD PR. This keeps the same semantics as the catalog does today in the sense that when a package is added to the catalog it is assumed that it will not be mutated. This is further enforced now by never exposing references to packages that are contained within the catalog.

Partially addresses #556 ; remaining work:

Port the SPDX JSON files relationships into the new artifact relationships (instead of doing do in the encoder)

github-actions · 2021-10-31T15:40:54Z

Benchmark Test Results

Benchmark results from the latest changes vs base branch

name                                                   old time/op    new time/op    delta
ImagePackageCatalogers/ruby-gemspec-cataloger-2          1.57ms ± 1%    1.85ms ± 3%   +18.25%  (p=0.008 n=5+5)
ImagePackageCatalogers/python-package-cataloger-2        3.62ms ± 9%    4.65ms ± 7%   +28.42%  (p=0.008 n=5+5)
ImagePackageCatalogers/javascript-package-cataloger-2     933µs ± 2%    1064µs ± 3%   +14.07%  (p=0.008 n=5+5)
ImagePackageCatalogers/dpkgdb-cataloger-2                1.01ms ± 2%    1.34ms ± 3%   +31.70%  (p=0.008 n=5+5)
ImagePackageCatalogers/rpmdb-cataloger-2                  957µs ± 2%    1136µs ± 4%   +18.70%  (p=0.008 n=5+5)
ImagePackageCatalogers/java-cataloger-2                  13.5ms ± 2%    15.4ms ± 4%   +14.68%  (p=0.008 n=5+5)
ImagePackageCatalogers/apkdb-cataloger-2                 1.39ms ± 2%    1.87ms ± 3%   +34.56%  (p=0.008 n=5+5)
ImagePackageCatalogers/go-module-binary-cataloger-2       746ns ± 1%    1939ns ± 1%  +159.81%  (p=0.008 n=5+5)

name                                                   old alloc/op   new alloc/op   delta
ImagePackageCatalogers/ruby-gemspec-cataloger-2           247kB ± 0%     286kB ± 0%   +15.73%  (p=0.008 n=5+5)
ImagePackageCatalogers/python-package-cataloger-2        1.10MB ± 0%    1.29MB ± 0%   +16.56%  (p=0.008 n=5+5)
ImagePackageCatalogers/javascript-package-cataloger-2     197kB ± 0%     211kB ± 0%    +7.10%  (p=0.008 n=5+5)
ImagePackageCatalogers/dpkgdb-cataloger-2                 228kB ± 0%     269kB ± 0%   +18.12%  (p=0.008 n=5+5)
ImagePackageCatalogers/rpmdb-cataloger-2                  221kB ± 0%     234kB ± 0%    +5.64%  (p=0.008 n=5+5)
ImagePackageCatalogers/java-cataloger-2                  3.23MB ± 0%    3.36MB ± 0%    +3.84%  (p=0.008 n=5+5)
ImagePackageCatalogers/apkdb-cataloger-2                 1.29MB ± 0%    1.34MB ± 0%    +4.30%  (p=0.008 n=5+5)
ImagePackageCatalogers/go-module-binary-cataloger-2        336B ± 0%      480B ± 0%   +42.86%  (p=0.008 n=5+5)

name                                                   old allocs/op  new allocs/op  delta
ImagePackageCatalogers/ruby-gemspec-cataloger-2           6.79k ± 0%     8.90k ± 0%   +31.08%  (p=0.008 n=5+5)
ImagePackageCatalogers/python-package-cataloger-2         26.3k ± 0%     38.3k ± 0%   +46.07%  (p=0.008 n=5+5)
ImagePackageCatalogers/javascript-package-cataloger-2     5.17k ± 0%     5.80k ± 0%   +12.33%  (p=0.008 n=5+5)
ImagePackageCatalogers/dpkgdb-cataloger-2                 6.65k ± 0%     8.51k ± 0%   +28.06%  (p=0.008 n=5+5)
ImagePackageCatalogers/rpmdb-cataloger-2                  6.53k ± 0%     7.09k ± 0%    +8.54%  (p=0.008 n=5+5)
ImagePackageCatalogers/java-cataloger-2                   58.9k ± 0%     66.8k ± 0%   +13.46%  (p=0.008 n=5+5)
ImagePackageCatalogers/apkdb-cataloger-2                  7.71k ± 0%    11.11k ± 0%   +44.05%  (p=0.016 n=5+4)
ImagePackageCatalogers/go-module-binary-cataloger-2        9.00 ± 0%     11.00 ± 0%   +22.22%  (p=0.008 n=5+5)

syft/artifact/relationship.go

cmd/packages.go

cmd/power_user_tasks.go

internal/anchore/import_package_sbom.go

internal/formats/spdx22json/to_format_model.go

internal/formats/syftjson/to_format_model.go

syft/format/decoder.go

wagoodman · 2021-11-08T21:42:42Z

I'm going to rebase since #606 was merged and has made this PR look a little crazy.

wagoodman · 2021-11-08T21:52:52Z

Since the package fingerprinting and SBOM data structure PRs have landed I think I'm ready to start tackling replacing ID assignment with the fingerprinting method. This implies considering possible removal of the pkg.Catalog --I'll see what shakes out.

wagoodman · 2021-11-08T22:14:59Z

One complexity that has been introduced that I want to note before I get too far: since the package ID is a function of the bytes on the pkg.Package struct (as to remain stable between multiple syft runs) and not a static string (such as a UUID), upon translating from one format to another we need to keep track of the string IDs from a static document and look up the object that they represent. The odd thing about this is that there is no guarantee that data is lossless during format conversion (say from syft JSON to SPDX) --in-fact it's a good bet that when going from syft-json to SPDX-json to syft-json again the IDs will change (since there is data loss)!

This extra complexity should remain only in the *.toSyftModel() functions for each format and not leak elsewhere.

luhring · 2021-11-08T22:38:13Z

One complexity that has been introduced that I want to note before I get too far: since the package ID is a function of the bytes on the pkg.Package struct [...], upon translating from one format to another we need to keep track of the string IDs from a static document and look up the object that they represent.

Question: Are we treating the static package ID values on any formats (that Syft can decode) as "okay to discard" during decoding? Specifically, I'm asking about more than using the IDs for things like discovering relationships, I'm asking about preserving the literal ID values themselves.

This is a really interesting perspective, and it's making me question an early opinion I had of not retaining static ID values on native Syft packages (in the business layer). As we think about using Syft as a library, I'm wondering if consumers of the library might have a reasonable expectation that they can find the original ID values (say, from there SPDX document that they Syft-decoded), within the business objects that Syft exposes in the library, for some task they're trying to accomplish. (Just wondering... I don't have an exact scenario in mind. Maybe some sort of policy logic.)

wagoodman · 2021-11-09T22:15:35Z

I'm wondering if consumers of the library might have a reasonable expectation that they can find the original ID values (say, from there SPDX document that they Syft-decoded)

With the last few commits in to prove out the PoC, I've been wondering the same thing.

As a <side note>... as of right now (at the time of this writing) there is no ID problem across multiple formats from a user perspective. That is, ID is not a field that is exposed in the existing CycloneDX and SPDX implementations (SPDX uses name/type which should change since this can create problems and we do not use the bom-ref field in the definition of a component, but this will change as we need component refs in the near future).
</side note>

We've changed ID to be behavior of a package and not a static (random) data value on a package. One nicety of a stabilizing ID based on contents is that the ID matches between multiple runs of analyzing the same source. The downside is that if the package data is lossy from converting from format-to-format (in the near future) then the ID will be different between documents.

I think we may have reached two mutually exclusive use cases here: one use case wanting ID stability based on "the definition of a package" and another use case wanting ID stability across different descriptions of the same basic package.

We cannot use the same field to handle both use cases. Either we will need to declare that one of these use cases is unsupported or we will need to split up responsibilities between two different fields. (anyone see a third option?)

luhring · 2021-11-10T12:58:52Z

That <side note> is a very good one to realize. 😄

I think we may have reached two mutually exclusive use cases here: one use case wanting ID stability based on "the definition of a package" and another use case wanting ID stability across different descriptions of the same basic package.

We cannot use the same field to handle both use cases. Either we will need to declare that one of these use cases is unsupported or we will need to split up responsibilities between two different fields. (anyone see a third option?)

Well said. My two cents: in general, I still favor the ID() approach where it's a stable identifier yielded from a deterministic function that considers the package's (or other artifact's) distinct identity. Maybe we start with this, and when (if?) the need arises for capturing other identifiers, we solve that in a designated part of the data structure (e.g. originalID or something). WDYT?

wagoodman · 2021-11-10T13:58:32Z

I landed on the same side of the fence as you @luhring --I think this is good enough for now, the new behavior is better than before, and when we have a specific use case that is raising a problem we can solve it then.

spiffcs · 2021-11-10T14:56:22Z

Maybe we start with this, and when (if?) the need arises for capturing other identifiers, we solve that in a designated part of the data structure (e.g. originalID or something). WDYT?

I don't think bifurcating to other designated spots for an originalID is what we want here. I think in future PR it will be important to identify the base structs/fields that each format share.

one use case wanting ID stability based on "the definition of a package" and another use case wanting ID stability across different descriptions of the same basic package.

If the definition of a package (before enoding to a format) is the same at the bottom of syft, then I believe there is a scenario where we fulfill both cases here so long as the generated ID happens BEFORE any format/presentation specific structs have a possibility of being hashed.

spiffcs · 2021-11-10T15:49:36Z

cmd/power_user.go

+	}
+}
+
+func mergeResults(cs ...<-chan artifact.Relationship) <-chan artifact.Relationship {


Nit: When dealing with more complex types like <-chan artifact.Relationship I find it easier to reason about the function signature if the return value is also declared (results <-chan artifact.Relationship).

If this breaks a convention we have in the codebase where we don't construct signatures that way feel free to ignore

I also try to do the same thing you are suggesting for small functions. Though in this case it isn't possible to keep the same signature and do this suggestion. The implementation creates a bidirectional channel that is encapsulated within mergeResults but a receive-only channel is returned (restricting callers from sending into the channel). If I were to use the signature to declare the return variable then I must change the return type to be a bidirectional channel, which isn't ideal.

wagoodman · 2021-11-10T16:23:49Z

re: #607 (comment) , I agree, the suggestion of keeping originalID probably isn't ideal --but we haven't made any decisions about that. What we're settling on is that we can keep the ID system as it is today (based on the object hash) and solve this later if we see a specific problem with that approach in the future.

If the definition of a package (before enoding to a format) is the same at the bottom of syft, then I believe there is a scenario where we fulfill both cases here so long as the generated ID happens BEFORE any format/presentation specific structs have a possibility of being hashed.

I'd have to think on this; it appears that what you're saying is true within the scope of generating a single document, but not necessarily true when converting from one format to another (something that isn't supported by syft yet anyway, so isn't a concern today, but could be in the future).

spiffcs

Comments - I'll dig in again later

spiffcs · 2021-11-10T15:50:08Z

cmd/power_user.go

+	}
+}
+
+func mergeResults(cs ...<-chan artifact.Relationship) <-chan artifact.Relationship {


If this breaks a convention we have in the codebase where we don't construct signatures that way feel free to ignore

internal/formats/common/spdxhelpers/license.go

spiffcs · 2021-11-10T16:01:45Z

syft/artifact/id.go

+	ID() ID
+}
+
+func DeriveID(obj interface{}) (ID, error) {


Some of the larger comments on this PR talk about formalizing the ID across formats. I think a more strict signature might help with this.

If we have this as interface what are all possible types within this project that could be passed here?
Can we limit it to some common field shared among all types?

I though about doing something along those lines, where there would be a IDFromLocation(...) and IDFromPackage(...) in the artifact package. The first problem is that this would introduce a cyclic dependency between package and artifact --ideally it should be the artifact package that is at the "root". I then realized that all of the ID implementations would be the same anyway (pass an object into Hash). For this reason I went the direction of making a single utility function (also, I renamed this in a later commit to IDFromHash(...)).

The largest con against this decision I see is that it would be possible to get an artifact.ID for something that doesn't represent an artifact. However, since artifact.ID is a string, there is nothing stopping anyone from creating artifact.IDs --that is, there is no enforcement through encapsulation for this.

It could be that we move this function to a separate package... maybe internal/hash? Then at least there wouldn't be any problems with cyclic dependencies.

spiffcs · 2021-11-10T16:13:50Z

syft/pkg/catalog.go

@@ -38,21 +41,26 @@ func (c *Catalog) PackageCount() int {
 }

 // Package returns the package with the given ID.
-func (c *Catalog) Package(id ID) *Package {
+func (c *Catalog) Package(id artifact.ID) *Package {


Do we return a copy now because we want to force the immutability of the catalog?

exactly 👍 I just updated the PR description a minute ago (after you made these comments) to reflect these thoughts

syft/pkg/catalog.go

spiffcs · 2021-11-10T16:16:34Z

syft/pkg/cataloger/apkdb/parse_apk_db.go

@@ -21,7 +23,7 @@ var _ common.ParserFn = parseApkDB

 // parseApkDb parses individual packages from a given Alpine DB file. For more information on specific fields
 // see https://wiki.alpinelinux.org/wiki/Apk_spec .
-func parseApkDB(_ string, reader io.Reader) ([]pkg.Package, error) {
+func parseApkDB(_ string, reader io.Reader) ([]pkg.Package, []artifact.Relationship, error) {


Are we adding relationships as a placeholder for now? On parse what relationships are we expecting to be able to generate?

Yeah, a necessary placeholder for now 👍

Since the generic cataloger has been enhanced to allow for parser functions to raise up artifact relationships, and the APK cataloger uses the generic cataloger, that meant that this function signature needed to be updated as well.

There may not be any relationships that the APK DB parser will raise up, but other parser functions may have the opportunity to raise up relationships (such as gemfile.lock and rpmdb parsers). I haven't double checked the underlying data available for APK yet, but APK might have package dependency data for us to leverage.

spiffcs · 2021-11-10T16:16:55Z

syft/pkg/cataloger/apkdb/parse_apk_db_test.go

@@ -775,7 +775,7 @@ func TestMultiplePackages(t *testing.T) {
 				}
 			}()

-			pkgs, err := parseApkDB(file.Name(), file)
+			pkgs, _, err := parseApkDB(file.Name(), file)


Probably TODO for future

I went back and forth on this. There are a lot of places that ignore relationship data in test results now that parser functions raise up artifact relationships. I made certain to wire up anywhere that could raise up relationship data in the future in the cataloging execution path even if the cataloger in question does not surface relationships. However, I did not update the tests since this PR doesn't actually add any new relationships.

My take: any PR that is enhancing one or more catalogers to discover relationships should come with tests that prove the new functionality works. The lack of tests on a PR I think would be more of a signal than anything else.

disregard... TODO statements here we come...

👍 628c2e4

spiffcs · 2021-11-10T16:21:55Z

syft/pkg/cataloger/common/cpe/generate.go

@@ -83,7 +83,7 @@ func candidateVendors(p pkg.Package) []string {
 	// allow * as a candidate. Note: do NOT allow Java packages to have * vendors.
 	switch p.Language {
 	case pkg.Ruby, pkg.JavaScript:
-		vendors.addValue("*")
+		vendors.addValue(wfn.Any)


Thanks... that one took a little while to find and is one of the main reasons why I beefed up the encode-decode cycle tests for all of this.

wagoodman · 2021-11-10T20:09:36Z

From an offline conversation: it looks like ID() on source.Location could be problematic when working with fields that either have a ref or do not. This PR opted out of this problem by ignoring the ref field (and VirutalPath), however, this won't help in situations where we index by source.Location but only have the RealPath and FilesystemID (when we port over the SPDX package-to-file relationship). I'll keep looking into this.

wagoodman · 2021-11-10T21:10:56Z

To account for the problem mentioned in #607 (comment) I've split out source.Location into a new source.Coordinates which holds the minimal information needed for determining "where" a file is at relative to a source. (I'll update the PR description)

I'll break this out into its own PR

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* migrate pkg.ID and pkg.Relationship to artifact package Signed-off-by: Alex Goodman <alex.goodman@anchore.com> * return relationships from tasks Signed-off-by: Alex Goodman <alex.goodman@anchore.com> * fix more tests Signed-off-by: Alex Goodman <alex.goodman@anchore.com> * add artifact.Identifiable by Identity() method Signed-off-by: Alex Goodman <alex.goodman@anchore.com> * fix linting Signed-off-by: Alex Goodman <alex.goodman@anchore.com> * remove catalog ID assignment Signed-off-by: Alex Goodman <alex.goodman@anchore.com> * adjust spdx helpers to use copy of packages Signed-off-by: Alex Goodman <alex.goodman@anchore.com> * stabilize package ID relative to encode-decode format cycles Signed-off-by: Alex Goodman <alex.goodman@anchore.com> * rename Identity() to ID() Signed-off-by: Alex Goodman <alex.goodman@anchore.com> * use zero value for nils in ID generation Signed-off-by: Alex Goodman <alex.goodman@anchore.com> * enable source.Location to be identifiable Signed-off-by: Alex Goodman <alex.goodman@anchore.com> * hoist up package relationship discovery to analysis stage Signed-off-by: Alex Goodman <alex.goodman@anchore.com> * update ownership-by-file-overlap relationship description Signed-off-by: Alex Goodman <alex.goodman@anchore.com> * add test reminders to put new relationships under test Signed-off-by: Alex Goodman <alex.goodman@anchore.com> * adjust PHP composer.lock parser function to return relationships Signed-off-by: Alex Goodman <alex.goodman@anchore.com> Signed-off-by: fsl <1171313930@qq.com>

* migrate pkg.ID and pkg.Relationship to artifact package Signed-off-by: Alex Goodman <alex.goodman@anchore.com> * return relationships from tasks Signed-off-by: Alex Goodman <alex.goodman@anchore.com> * fix more tests Signed-off-by: Alex Goodman <alex.goodman@anchore.com> * add artifact.Identifiable by Identity() method Signed-off-by: Alex Goodman <alex.goodman@anchore.com> * fix linting Signed-off-by: Alex Goodman <alex.goodman@anchore.com> * remove catalog ID assignment Signed-off-by: Alex Goodman <alex.goodman@anchore.com> * adjust spdx helpers to use copy of packages Signed-off-by: Alex Goodman <alex.goodman@anchore.com> * stabilize package ID relative to encode-decode format cycles Signed-off-by: Alex Goodman <alex.goodman@anchore.com> * rename Identity() to ID() Signed-off-by: Alex Goodman <alex.goodman@anchore.com> * use zero value for nils in ID generation Signed-off-by: Alex Goodman <alex.goodman@anchore.com> * enable source.Location to be identifiable Signed-off-by: Alex Goodman <alex.goodman@anchore.com> * hoist up package relationship discovery to analysis stage Signed-off-by: Alex Goodman <alex.goodman@anchore.com> * update ownership-by-file-overlap relationship description Signed-off-by: Alex Goodman <alex.goodman@anchore.com> * add test reminders to put new relationships under test Signed-off-by: Alex Goodman <alex.goodman@anchore.com> * adjust PHP composer.lock parser function to return relationships Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

wagoodman force-pushed the single-sbom-document branch 2 times, most recently from bf6a2cd to 4757c25 Compare November 2, 2021 17:39

wagoodman mentioned this pull request Nov 2, 2021

Introduce a single SBOM document #606

Merged

wagoodman force-pushed the single-sbom-document branch from 4757c25 to 95bd1fd Compare November 2, 2021 18:24

wagoodman requested a review from a team November 2, 2021 18:29

wagoodman self-assigned this Nov 2, 2021

luhring reviewed Nov 2, 2021

View reviewed changes

syft/artifact/relationship.go Outdated Show resolved Hide resolved

spiffcs force-pushed the single-sbom-document branch 3 times, most recently from 19b9013 to 197c27b Compare November 3, 2021 18:41

spiffcs reviewed Nov 5, 2021

View reviewed changes

Base automatically changed from single-sbom-document to main November 5, 2021 14:05

wagoodman force-pushed the first-class-relationships branch 2 times, most recently from e476f35 to 69d2b1b Compare November 8, 2021 21:46

wagoodman force-pushed the first-class-relationships branch from d486512 to 52adfcb Compare November 9, 2021 21:53

spiffcs reviewed Nov 10, 2021

View reviewed changes

wagoodman marked this pull request as ready for review November 10, 2021 17:18

wagoodman force-pushed the first-class-relationships branch from 1d26197 to 628c2e4 Compare November 10, 2021 21:16

wagoodman mentioned this pull request Nov 10, 2021

Introduce minimal source coordinates #623

Merged

wagoodman added 15 commits November 16, 2021 10:53

migrate pkg.ID and pkg.Relationship to artifact package

0a983ad

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

return relationships from tasks

348e84d

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

fix more tests

c9b80e7

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

add artifact.Identifiable by Identity() method

e1738d8

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

fix linting

d33e9c4

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

remove catalog ID assignment

40e3248

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

adjust spdx helpers to use copy of packages

19aba4e

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

stabilize package ID relative to encode-decode format cycles

84d000e

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

rename Identity() to ID()

c21d3d3

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

use zero value for nils in ID generation

0c30443

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

enable source.Location to be identifiable

15e772b

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

hoist up package relationship discovery to analysis stage

86bed99

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

update ownership-by-file-overlap relationship description

3f2126e

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

add test reminders to put new relationships under test

2b12d13

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

adjust PHP composer.lock parser function to return relationships

77406a7

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

wagoodman force-pushed the first-class-relationships branch from 628c2e4 to 77406a7 Compare November 16, 2021 15:56

spiffcs approved these changes Nov 16, 2021

View reviewed changes

wagoodman merged commit ef627d8 into main Nov 16, 2021

wagoodman deleted the first-class-relationships branch November 16, 2021 19:14

wagoodman mentioned this pull request Nov 17, 2021

Migrate SPDX-JSON relationships to SBOM model #634

Merged

wagoodman mentioned this pull request Dec 15, 2021

Cataloging large images is taking too long #696

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

First class relationships #607

First class relationships #607

wagoodman commented Oct 31, 2021 •

edited

github-actions bot commented Oct 31, 2021 •

edited

wagoodman commented Nov 8, 2021 •

edited

wagoodman commented Nov 8, 2021

wagoodman commented Nov 8, 2021

luhring commented Nov 8, 2021

wagoodman commented Nov 9, 2021 •

edited

luhring commented Nov 10, 2021

wagoodman commented Nov 10, 2021

spiffcs commented Nov 10, 2021

spiffcs Nov 10, 2021

spiffcs Nov 10, 2021

wagoodman Nov 10, 2021

wagoodman commented Nov 10, 2021

spiffcs left a comment

spiffcs Nov 10, 2021

spiffcs Nov 10, 2021

wagoodman Nov 10, 2021

spiffcs Nov 10, 2021

wagoodman Nov 10, 2021 •

edited

spiffcs Nov 10, 2021

wagoodman Nov 10, 2021

wagoodman Nov 10, 2021

spiffcs Nov 10, 2021

wagoodman Nov 10, 2021

wagoodman Nov 10, 2021

wagoodman Nov 10, 2021

spiffcs Nov 10, 2021

wagoodman Nov 10, 2021

wagoodman commented Nov 10, 2021 •

edited

wagoodman commented Nov 10, 2021 •

edited

First class relationships #607

First class relationships #607

Conversation

wagoodman commented Oct 31, 2021 • edited

github-actions bot commented Oct 31, 2021 • edited

Benchmark Test Results

wagoodman commented Nov 8, 2021 • edited

wagoodman commented Nov 8, 2021

wagoodman commented Nov 8, 2021

luhring commented Nov 8, 2021

wagoodman commented Nov 9, 2021 • edited

luhring commented Nov 10, 2021

wagoodman commented Nov 10, 2021

spiffcs commented Nov 10, 2021

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

wagoodman commented Nov 10, 2021

spiffcs left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

wagoodman Nov 10, 2021 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

wagoodman commented Nov 10, 2021 • edited

wagoodman commented Nov 10, 2021 • edited

wagoodman commented Oct 31, 2021 •

edited

github-actions bot commented Oct 31, 2021 •

edited

wagoodman commented Nov 8, 2021 •

edited

wagoodman commented Nov 9, 2021 •

edited

wagoodman Nov 10, 2021 •

edited

wagoodman commented Nov 10, 2021 •

edited

wagoodman commented Nov 10, 2021 •

edited