test: Update Python interop tests to python-tuf v1.0.0 #228

znewman01 · 2022-03-02T02:13:29Z

Fixes #221

znewman01 · 2022-03-02T02:29:55Z

CC @jku to make sure the new python-tuf code is right; in particular:

there's no built-in support for consistent snapshots, right?
is there a better way to make sure that timestamp.json has correct hashes/length than what I'm doing?

My tests are passing, but some others are not passing due to the ErrWrongID change; would appreciate feedback. Core issue:

e82b56b started ignoring ErrWrongID errors because of TAP-12
TAP-12 seemingly got implemented in python-tuf v1.0.0
however the change from (1) doesn't actually import the key! it just silently ignores it

Is the right thing to do:

remove ErrWrongID generally?
have a more-convoluted check, like "if keyid is provided check it but otherwise don't bother"?

jku

Left some comments hopefully they are helpful

The metadata generation is quite verbose as is visible here: I hope we can improve on that soon, I'll try to remember to send a patch here when that happens.

there's no built-in support for consistent snapshots, right?

I think the metadata you now produce is completely valid consistent_snapshot (the non-consistent version seems a bit incorrect as root is not versioned but I commented on that in code). Let me know if there's anything that seems missing.

It's true we don't currently handle target files themselves in any way so you will need to produce the hash prefixed filename yourself when saving the file, that is a bit annoying: I left a comment in the code.

is there a better way to make sure that timestamp.json has correct hashes/length than what I'm doing?

this is an oversight in python-tuf: we've not been using the optional hashes in meta dicts

TAP-12 seemingly got implemented in python-tuf v1.0.0

Maybe more accurate to say Metadata API does not enforce or test that the id actually is the hex digest of the canonical form: it's possible that as an accidental result of that we are producing metadata that is not strictly conformant with that current requirement.

client/python_interop/testdata/python-tuf-v1.0.0/generate.py

znewman01 · 2022-03-02T20:49:48Z

@jku:

Left some comments hopefully they are helpful

Yes, thank you for the quick review 😄 great context

Maybe more accurate to say Metadata API does not enforce or test that the id actually is the hex digest of the canonical form: it's possible that as an accidental result of that we are producing metadata that is not strictly conformant with that current requirement.

ACK, makes sense. I will consider this a python-tuf bug and track python-tuf#1894.

However, it seems like it should be okay to ignore the keyIDs as a client and barring objections I will update the PR to allow this.

coveralls · 2022-03-02T21:19:48Z

Pull Request Test Coverage Report for Build 1924696848

8 of 13 (61.54%) changed or added relevant lines in 4 files are covered.
6 unchanged lines in 4 files lost coverage.
Overall coverage decreased (-0.02%) to 70.215%

Changes Missing Coverage	Changed/Added Lines	%
repo.go	1	0.0%
client/client.go	2	0.0%
verify/errors.go	2	0.0%

Files with Coverage Reduction	New Missed Lines	%
client/client.go	1	81.41%
repo.go	1	73.37%
verify/errors.go	1	0%
verify/db.go	3	85.53%

Totals
Change from base Build 1831480097:	-0.02%
Covered Lines:	2157
Relevant Lines:	3072

💛 - Coveralls

verify/db.go

mnm678 · 2022-03-04T16:49:32Z

TAP-12 seemingly got implemented in python-tuf v1.0.0

Maybe more accurate to say Metadata API does not enforce or test that the id actually is the hex digest of the canonical form: it's possible that as an accidental result of that we are producing metadata that is not strictly conformant with that current requirement.

With the python-tuf implementation, I think that we can move TAP 12 to accepted pretty quickly. Unless there are strong objections to the TAP that will lead to major changes (which I haven't seen), I'd argue for future-proofing go-tuf by implementing TAP 12 now. It may make the implementation temporarily out of sync with the specification, but not in a major way, and not in a way that affects security (as described in more detail in the TAP).

znewman01 · 2022-03-04T22:52:38Z

Filed #232 for TAP-12.

I think we should probably block this PR on that issue. That would make sure we don't half-implement TAP-12 and accidentally fail-open.

joshuagl · 2022-06-21T16:08:05Z

As #232 is merged can we rebase this and get it reviewed @znewman01?

znewman01 · 2022-06-21T16:41:49Z

Yup! Might be a week-ish until I get around to it.

joshuagl · 2022-06-21T18:55:26Z

np, let us know when you're ready for review!

asraa · 2022-08-16T13:48:05Z

@znewman01 mind if I update this?

znewman01 · 2022-08-16T13:51:49Z

Go for it! Let me know if you don’t find time,it had finally popped to the top of my TODO list this week but happy to have you take it off my hands

…

On Aug 16, 2022, at 9:48 AM, asraa ***@***.***> wrote: @znewman01 mind if I update this? — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.

asraa

@znewman01 rebased, tests passing

asraa · 2022-08-16T14:55:36Z

@jku @mnm678 could you PTAL? TAP-12 key ID removal is done, and so is some minor go-tuf snapshot length/hashes presence bug is resolved too.

znewman01 · 2022-08-17T17:00:44Z

LGTM though I can't review it since it's my PR

trishankatdatadog · 2022-08-17T17:26:24Z

I'll leave it to @jku since he's best suited to review this one

jku · 2022-08-18T20:45:25Z

I'll leave it to @jku since he's best suited to review this one

Sorry, I'm unlikely to have a real look in the next weeks.

client/python_interop/testdata/python-tuf-v1.0.0/generate.py

Signed-off-by: Zachary Newman <z@znewman.net> Signed-off-by: Asra Ali <asraa@google.com>

A few changes, besides pointing at the new directory: - the python-tuf repo says "High-level support for implementing repository operations is planned but not yet provided;" this includes consistent snapshot, so we'll add a consistent snapshot test at that point. - make a new tmp dir for each Python client test (it gets confused otherwise) - the new python-tuf client downloads `foo/bar.txt` to `targets/foo%2Fbar.txt` so check for it there Signed-off-by: Zachary Newman <z@znewman.net> Signed-off-by: Asra Ali <asraa@google.com>

Signed-off-by: Zachary Newman <z@znewman.net> Signed-off-by: Asra Ali <asraa@google.com>

Signed-off-by: Asra Ali <asraa@google.com>

trishankatdatadog · 2022-08-26T17:35:43Z

I made an exception for the DCO this time.

znewman01 force-pushed the interop branch 2 times, most recently from d31016c to d72fa29 Compare March 2, 2022 02:24

jku reviewed Mar 2, 2022

View reviewed changes

client/python_interop/testdata/python-tuf-v1.0.0/generate.py Outdated Show resolved Hide resolved

client/python_interop/testdata/python-tuf-v1.0.0/generate.py Outdated Show resolved Hide resolved

client/python_interop/testdata/python-tuf-v1.0.0/generate.py Show resolved Hide resolved

This was referenced Mar 2, 2022

MetaFile is missing from_bytes() or equivalent theupdateframework/python-tuf#1893

Closed

Reconsider keyid "compliance" theupdateframework/python-tuf#1894

Closed

znewman01 mentioned this pull request Mar 2, 2022

go-tuf requires snapshot.json length/hashes to be set in timestamp meta #229

Closed

asraa reviewed Mar 2, 2022

View reviewed changes

verify/db.go Outdated Show resolved Hide resolved

znewman01 mentioned this pull request Mar 4, 2022

Implement TAP-12 for future-proofing #232

Closed

4 tasks

This was referenced Apr 28, 2022

chore(deps): bump tuf from 0.20.0 to 1.1.0 #279

Closed

chore(deps): bump securesystemslib from 0.21.0 to 0.23.0 #278

Closed

znewman01 mentioned this pull request Jun 9, 2022

build: update go version to 1.18 #314

Merged

1 task

rdimitrov mentioned this pull request Jun 10, 2022

chore(deps): bump Python from 3.6 to 3.10 #318

Merged

2 tasks

asraa force-pushed the interop branch 2 times, most recently from 59120cb to 000d999 Compare August 16, 2022 14:37

asraa changed the title ~~Update Python interop tests to python-tuf v1.0.0~~ test: Update Python interop tests to python-tuf v1.0.0 Aug 16, 2022

asraa mentioned this pull request Aug 16, 2022

chore(deps): bump tuf from 0.20.0 to 2.0.0 #356

Closed

asraa reviewed Aug 16, 2022

View reviewed changes

mnm678 approved these changes Aug 16, 2022

View reviewed changes

asraa requested a review from jku August 17, 2022 15:51

trishankatdatadog approved these changes Aug 26, 2022

View reviewed changes

client/python_interop/testdata/python-tuf-v1.0.0/generate.py Show resolved Hide resolved

client/python_interop/testdata/python-tuf-v1.0.0/generate.py Show resolved Hide resolved

znewman01 and others added 8 commits August 26, 2022 12:46

Add python-tuf-v1.0.0 client/generator

ed09066

Signed-off-by: Zachary Newman <z@znewman.net> Signed-off-by: Asra Ali <asraa@google.com>

Update python_interop Makefile to generate v1.0.0

e9c727d

Signed-off-by: Zachary Newman <z@znewman.net> Signed-off-by: Asra Ali <asraa@google.com>

Remove old python_interop for python-tuf v0.11.1

3375b5d

Signed-off-by: Zachary Newman <z@znewman.net> Signed-off-by: Asra Ali <asraa@google.com>

Add consistent snapshot (generator + test)

9b578de

Signed-off-by: Asra Ali <asraa@google.com>

Regenerate python_interop testdata

15d9d48

Signed-off-by: Asra Ali <asraa@google.com>

update testadata and requirements

bea4318

Signed-off-by: Asra Ali <asraa@google.com>

remove addition of snapshot meta in timestamp

674204e

Signed-off-by: Asra Ali <asraa@google.com>

trishankatdatadog force-pushed the interop branch from fa71b4d to 674204e Compare August 26, 2022 16:46

trishankatdatadog merged commit d7ff71b into theupdateframework:master Aug 26, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

test: Update Python interop tests to python-tuf v1.0.0 #228

test: Update Python interop tests to python-tuf v1.0.0 #228

znewman01 commented Mar 2, 2022

znewman01 commented Mar 2, 2022

jku left a comment

znewman01 commented Mar 2, 2022

coveralls commented Mar 2, 2022 •

edited

mnm678 commented Mar 4, 2022

znewman01 commented Mar 4, 2022

joshuagl commented Jun 21, 2022

znewman01 commented Jun 21, 2022

joshuagl commented Jun 21, 2022

asraa commented Aug 16, 2022

znewman01 commented Aug 16, 2022 via email

asraa left a comment

asraa commented Aug 16, 2022

znewman01 commented Aug 17, 2022

trishankatdatadog commented Aug 17, 2022

jku commented Aug 18, 2022 •

edited

trishankatdatadog commented Aug 26, 2022

test: Update Python interop tests to python-tuf v1.0.0 #228

test: Update Python interop tests to python-tuf v1.0.0 #228

Conversation

znewman01 commented Mar 2, 2022

znewman01 commented Mar 2, 2022

jku left a comment

Choose a reason for hiding this comment

znewman01 commented Mar 2, 2022

coveralls commented Mar 2, 2022 • edited

Pull Request Test Coverage Report for Build 1924696848

💛 - Coveralls

mnm678 commented Mar 4, 2022

znewman01 commented Mar 4, 2022

joshuagl commented Jun 21, 2022

znewman01 commented Jun 21, 2022

joshuagl commented Jun 21, 2022

asraa commented Aug 16, 2022

znewman01 commented Aug 16, 2022 via email

asraa left a comment

Choose a reason for hiding this comment

asraa commented Aug 16, 2022

znewman01 commented Aug 17, 2022

trishankatdatadog commented Aug 17, 2022

jku commented Aug 18, 2022 • edited

trishankatdatadog commented Aug 26, 2022

coveralls commented Mar 2, 2022 •

edited

jku commented Aug 18, 2022 •

edited