-
Notifications
You must be signed in to change notification settings - Fork 13
/
root_signing.py
453 lines (345 loc) · 18.4 KB
/
root_signing.py
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
# -*- coding: utf-8 -*-
""" conda_content_trust.root_signing
This module contains functions that sign data in an OpenPGP-compliant (i.e.
GPG-friendly) way. Root metadata may be signed in this manner. Functions that
perform simpler, direct signing using raw ed25519 keys are provided in
conda_content_trust.signing instead.
This library takes advantage of the securesystemslib library for its gpg
signing interface.
Function Manifest for this Module:
sign_via_gpg # requires securesystemslib
sign_root_metadata_via_gpg # requires securesystemslib
fetch_keyval_from_gpg # requires securesystemslib
These two functions are provided only for testing purpose and are not part of
the API for this module:
_gpg_pubkey_in_ssl_format
_verify_gpg_sig_using_ssl # requires securesystemslib
Note that there is a function in conda_content_trust.authentication that verifies these
signatures without requiring securesystemslib.
"""
# Python2 Compatibility
from __future__ import absolute_import, division, print_function, unicode_literals
# std libs
import json
#import binascii # for binascii.unhexlify / hexlify
#import struct # for struct.pack
# dependencies
# For ed25519 signing operations and hashing
import cryptography.hazmat.primitives.asymmetric.ed25519# as ed25519
import cryptography.hazmat.primitives.hashes# as pyca_crypto_hashes
import cryptography.hazmat.backends# as pyca_crypto_backends
import cryptography.exceptions# as pyca_crypto_exceptions
# securesystemslib is an optional dependency, and required only for signing
# root metadata via GPG. Verification of those signatures, and signing other
# metadata with raw ed25519 signatures, does not require securesystemslib.
try:
import securesystemslib.gpg.functions as gpg_funcs
import securesystemslib.formats
SSLIB_AVAILABLE = True
except ImportError:
SSLIB_AVAILABLE = False
# this codebase
from .common import (
canonserialize, load_metadata_from_file, write_metadata_to_file,
is_a_signable,
checkformat_gpg_fingerprint, checkformat_hex_key,
checkformat_gpg_signature, checkformat_byteslike,
PrivateKey, PublicKey, checkformat_key)
def sign_via_gpg(data_to_sign, gpg_key_fingerprint, include_fingerprint=False):
"""
<Purpose>
This is an alternative to the conda_content_trust.common.PrivateKey.sign() method, for
use with OpenPGP keys, allowing us to use protected keys in YubiKeys
(which provide an OpenPGP interface) to sign data.
The signature is not simply over data_to_sign, as is the case with the
PrivateKey.sign() function, but over an expanded payload with
metadata about the signature to be signed, as specified by the OpenPGP
standard (RFC 4880). See data_to_sign and Security Note below.
This process is nominally deterministic, but varies with the precise
time, since there is a timestamp added by GPG into the signed payload.
Nonetheless, this process does not depend at any point on the ability
to generate random data (unlike key generation).
This function requires securesystemslib, which is otherwise an optional
dependency.
<Arguments>
data_to_sign
The raw bytes of interest that will be signed by GPG. Note that
pursuant to the OpenPGP standard, GPG will add to this data:
specifically, it includes metadata about the signature that is
about to be made into the data that will be signed. We do not care
about that metadata, and we do not want to burden signature
verification with its processing, so we essentially ignore it.
This should have negligible security impact, but for more
information, see "A note on security" below.
gpg_key_fingerprint
This is a (fairly) unique identifier for an OpenPGP key pair.
Also Known as a "long" GPG keyid, a GPG fingerprint is
40-hex-character string representing 20 bytes of raw data, the
SHA-1 hash of a collection of the GPG key's properties.
Internally, GPG uses the key fingerprint to identify keys the
client knows of.
Note that an OpenPGP public key is a larger object identified by a
fingerprint. GPG public keys include two things, from our
perspective:
- the raw bytes of the actual cryptographic key
(in our case the 32-byte value referred to as "q" for an ed25519
public key)
- lots of data that is totally extraneous to us, including a
timestamp, some representations of relationships with other keys
(subkeys, signed-by lists, etc.), Superman's real name
(see also https://bit.ly/38GcaGj), potential key revocations,
etc.
We do not care about this extra data because we are using the
OpenPGP standard not for its key-to-key semantics or any element
of its Public Key Infrastructure features (revocation, vouching
for other keys, key relationships, etc.), but simply as a means
of asking YubiKeys to sign data for us, with ed25519 keys whose
raw public key value ("q") we know to expect.
<Returns>
Returns a dictionary representing a GPG signature. This is similar to
but not *quite* the same as
securesystemslib.formats.GPG_SIGNATURE_SCHEMA (which uses 'keyid'
as the key for the fingerprint, instead of 'gpg_key_fingerprint').
Specifically, this looks like:
{'gpg_key_fingerprint': <gpg key fingerprint>,
'other_headers': <extra data mandated in OpenPGP signatures>,
'signature': <ed25519 signature, 64 bytes as 128 hex chars>}
This is unlike conda_content_trust.signing.sign(), which simply returns 64 bytes of raw
ed25519 signature.
<Security Note>
A note on the security implications of this treatment of OpenPGP
signatures:
TL;DR:
It is NOT easier for an attacker to find a collision; however, it
IS easier, IF an attacker CAN find a collision, to do so in a way
that presents a specific, arbitrary payload.
Note that pursuant to the OpenPGP standard, GPG will add to the data we
ask it to sign (data_to_sign) before signing it. Specifically, GPG will
add, to the payload-to-be-signed, OpenPGP metadata about the signature
it is about to create. We do not care about that metadata, and we do
not want to burden signature verification with its processing (that is,
we do not want to use GPG to verify these signatures; conda will do
that with simpler code). As a result, we will ignore this data when
parsing the signed payload. This will mean that there will be many
different messages that have the same meaning to us:
signed:
<some raw data we send to GPG: 'ABCDEF...'>
<some data GPG adds in: '123456...'>
Since we will not be processing the '123456...' above, '654321...'
would have the same effect: as long as the signature is verified,
we don't care what's in that portion of the payload.
Since there are many, many payloads that mean the same thing to us, an
attacker has a vast space of options all with the same meaning to us in
which to search for (effectively) a useful SHA256 hash collision to
find different data that says something *specific* and still
*succeeds* in signature verification using the same signature.
While that is not ideal, it is difficult enough simply to find a SHA256
collision that this is acceptable.
"""
if not SSLIB_AVAILABLE:
# TODO✅: Consider a missing-optional-dependency exception class.
raise Exception(
'sign_via_gpg requires the securesystemslib library, which '
'appears to be unavailable.')
# Argument validation
checkformat_gpg_fingerprint(gpg_key_fingerprint)
checkformat_byteslike(data_to_sign)
# try:
# full_gpg_pubkey = gpg_funcs.export_pubkey(gpg_key_fingerprint)
# except securesystemslib.gpg.exceptions.KeyNotFoundError as e:
# raise Exception( # TODO✅: Consider an appropriate error class.
# 'The GPG application reported that it is not aware of a key '
# 'with the fingerprint provided ("' + str(gpg_key_fingerprint) +
# '"). You may need to import the given key.')
sig = gpg_funcs.create_signature(data_to_sign, gpg_key_fingerprint)
# # 💣💥 Debug only.
# # 💣💥 Debug only.
# assert gpg_funcs.verify_signature(sig, full_gpg_pubkey, data_to_sign)
# securesystemslib.gpg makes use of the GPG key fingerprint. We don't
# care about that as much -- we want to use the raw ed25519 public key
# value to refer to the key in a manner consistent with the way we refer to
# non-GPG (non-OpenPGP) keys.
keyval = fetch_keyval_from_gpg(gpg_key_fingerprint)
# ssl gpg sigs look like this:
#
# {'keyid': <gpg key fingerprint>,
# 'other_headers': <extra data mandated in OpenPGP signatures>,
# 'signature': <actual ed25519 signature, 64 bytes as 128 hex chars>}
#
# We want to store the real public key instead of just the gpg key
# fingerprint, so we add that, and we'll rename keyid to
# gpg_key_fingerprint. That gives us:
#
# {'gpg_key_fingerprint': <gpg key fingerprint>,
# 'other_headers': <extra data mandated in OpenPGP signatures>,
# 'signature': <actual ed25519 signature, 64 bytes as 128 hex chars>}
#
# sig['key'] = keyval # q, the 32-byte raw ed25519 public key value, expressed as 64 hex characters
# The OpenPGP Fingerprint of the OpenPGP key used to sign. This is not
# required for verification, but it's useful for debugging and for
# root keyholder convenience. So it's optional.
if include_fingerprint:
sig['see_also'] = sig['keyid'] # strictly not needed, useful for debugging; 20-byte sha1 gpg key identifier per OpenPGP spec, expressed as 40 hex characters
del sig['keyid']
return sig
# TODO✅: Rename this to sign_root_metadata_via_gpg and rename
# the old sign_root_metadata_via_gpg to sign_root_metadata_file_via_gpg
def sign_root_metadata_dict_via_gpg(root_signable, gpg_key_fingerprint):
# Signs root_signable in place, returns nothing.
if not SSLIB_AVAILABLE:
# TODO✅: Consider a missing-optional-dependency exception class.
raise Exception(
'sign_root_metadata_via_gpg requires the securesystemslib library, which '
'appears to be unavailable.')
# Make sure it's the right format.
if not is_a_signable(root_signable):
raise TypeError(
'Expected a signable dictionary; the given file ' +
str(root_md_fname) + ' failed the check.')
# TODO: Add root-specific checks.
# Canonicalize and serialize the data, putting it in the form we expect to
# sign over. Note that we'll canonicalize and serialize the whole thing
# again once the signatures have been added.
data_to_sign = canonserialize(root_signable['signed'])
# sig_dict, pgp_pubkey = sign_via_gpg(data_to_sign, gpg_key_fingerprint)
sig_dict = sign_via_gpg(data_to_sign, gpg_key_fingerprint)
# sig_dict looks like this:
# {'keyid': 'f075dd2f6f4cb3bd76134bbb81b6ca16ef9cd589',
# 'other_headers': '04001608001d162104f075dd2f6f4cb3bd76134bbb81b6ca16ef9cd58905025dbc3e68',
# 'signature': '29282a8fe75871f9d4cf10a5a9e8d92303f8c361ce4b474a0ce641c9b8a74e4baaf810cc383af318a8e21cbe252789c2c30894d94e8b0288c3c45ceacf6c1d0c'}
# pgp_pubkey looks like this:
# {'creation_time': 1571411344,
# 'hashes': ['pgp+SHA2'],
# 'keyid': 'f075dd2f6f4cb3bd76134bbb81b6ca16ef9cd589',
# 'keyval': {'private': '',
# 'public': {'q': 'bfbeb6554fca9558da7aa05c5e9952b7a1aa3995dede93f3bb89f0abecc7dc07'}},
# 'method': 'pgp+eddsa-ed25519',
# 'type': 'eddsa'}
# securesystemslib.gpg makes use of the GPG key fingerprint. We don't
# care about that as much -- we want to use the raw ed25519 public key
# value to refer to the key in a manner consistent with the way we refer to
# non-GPG (non-OpenPGP) keys.
# raw_pubkey = pgp_pubkey['keyval']['public']['q']
raw_pubkey = fetch_keyval_from_gpg(gpg_key_fingerprint)
# non-GPG signing here would look like this:
# signature_as_hexstr = serialize_and_sign(signable['signed'], private_key)
# public_key_as_hexstr = binascii.hexlify(key_to_bytes(
# private_key.public_key())).decode('utf-8')
# TODO: ✅⚠️ Log a warning in whatever conda's style is (or conda-build):
#
# if public_key_as_hexstr in signable['signatures']:
# warn( # replace: log, 'warnings' module, print statement, whatever
# 'Overwriting existing signature by the same key on given '
# 'signable. Public key: ' + public_key + '.')
# Add signature in-place.
root_signable['signatures'][raw_pubkey] = sig_dict
return root_signable
def sign_root_metadata_via_gpg(root_md_fname, gpg_key_fingerprint):
"""
# TODO✅: Proper docstring:
# This is a higher-level function than sign_via_gpg, including code that
# deals with the filesystem. It is not actually limited to root metadata,
# and SHOULD BE RENAMED.
"""
# Read in json
root_signable = load_metadata_from_file(root_md_fname)
root_signable = sign_root_metadata_dict_via_gpg(root_signable, gpg_key_fingerprint)
# TODO: Consider removing write_metadata_to_file. It might be better for
# readers to see the canonserialize() call being made (again) here,
# and it's not that much longer....
write_metadata_to_file(root_signable, root_md_fname)
def fetch_keyval_from_gpg(fingerprint):
"""
Retrieve the underlying 32-byte raw ed25519 public key for a GPG key.
Given a GPG key fingerprint (40-character hex string), retrieve the GPG
key, parse it, and return "q", the 32-byte ed25519 key value.
This takes advantage of the GPG key parser in securesystemslib.
"""
if not SSLIB_AVAILABLE:
# TODO✅: Consider a missing-optional-dependency exception class.
raise Exception(
'fetch_keyval_from_gpg requires the securesystemslib library, which '
'appears to be unavailable.')
checkformat_gpg_fingerprint(fingerprint)
key_parameters = gpg_funcs.export_pubkey(fingerprint)
return key_parameters['keyval']['public']['q']
def _verify_gpg_sig_using_ssl(signature, gpg_key_fingerprint, key_value, data):
"""
THIS IS PROVIDED ONLY FOR TESTING PURPOSES.
We will verify signatures using our own code in conda_content_trust.authentication, not
by using the securesystemslib.gpg.functions.verify_signature call that
sits here.
Wraps securesystemslib.gpg.functions.verify_signature. to format the
arguments in a manner ssl will like (i.e. conforming to
securesystemslib.formats.GPG_SIGNATURE_SCHEMA).
"""
if not SSLIB_AVAILABLE:
# TODO✅: Consider a missing-optional-dependency exception class.
raise Exception(
'verifygpg_sig_using_ssl requires the securesystemslib '
'library, which appears to be unavailable.')
checkformat_key(key_value)
# This function validates these two args in the process of formatting them.
ssl_format_key = gpg_pubkey_in_ssl_format(gpg_key_fingerprint, key_value)
securesystemslib.formats.GPG_SIGNATURE_SCHEMA.check_match(signature)
securesystemslib.formats._GPG_ED25519_PUBKEY_SCHEMA.check_match(
ssl_format_key)
# TODO: ✅ Validate sig (ssl-format gpg sig dict) and content (bytes).
# Note: if we change the signature format to deviate from what ssl uses,
# then we need to correct it here if we're going to use ssl.
validity = gpg_funcs.verify_signature(signature, ssl_format_key, data)
return validity
def _gpg_pubkey_in_ssl_format(fingerprint, q):
"""
THIS IS PROVIDED ONLY FOR TESTING PURPOSES.
We do not need to convert pubkeys to securesystemslib's format, except to
try out securesystemslib's gpg signature verification (which we use only
for comparison during testing).
Given a GPG key fingerprint (40 hex characters) and a q value (64 hex
characters representing a 32-byte ed25519 public key raw value), produces a
key object in a format that securesystemslib expects, so that we can use
securesystemslib.gpg.functions.verify_signature for part of the GPG
signature verification. For our purposes, this means that we should
produce a dictionary conforming to
securesystemslib.formats._GPG_ED25519_PUBKEY_SCHEMA.
If securesystemslib.formats._GPG_ED25519_PUBKEY_SCHEMA changes, those
changes will likely need to be reflected here.
Example value produced:
{
'type': 'eddsa',
'method': 'pgp+eddsa-ed25519',
'hashes': ['pgp+SHA2'],
'keyid': 'F075DD2F6F4CB3BD76134BBB81B6CA16EF9CD589',
'keyval': {
'public': {'q': 'bfbeb6554fca9558da7aa05c5e9952b7a1aa3995dede93f3bb89f0abecc7dc07'},
'private': ''}
}
}
"""
checkformat_gpg_fingerprint(fingerprint)
checkformat_hex_key(q)
ssl_format_key = {
'type': 'eddsa',
'method': securesystemslib.formats.GPG_ED25519_PUBKEY_METHOD_STRING,
'hashes': [securesystemslib.formats.GPG_HASH_ALGORITHM_STRING],
'keyid': fingerprint,
'keyval': {'private': '', 'public': {'q': q}}
}
return ssl_format_key
# def _gpgsig_to_sslgpgsig(gpg_sig):
#
# conda_content_trust.common.checkformat_gpg_signature(gpg_sig)
#
# return {
# 'keyid': copy.deepcopy(gpg_sig['key_fingerprint']),
# 'other_headers': copy.deepcopy(gpg_sig[other_headers]),
# 'signature': copy.deepcopy(gpg_sig['signature'])}
# def _sslgpgsig_to_gpgsig(ssl_gpg_sig):
#
# securesystemslib.formats.GPG_SIGNATURE_SCHEMA.check_match(ssl_gpg_sig)
#
# return {
# 'key_fingerprint': copy.deepcopy(ssl_gpg_sig['keyid']),
# 'other_headers': copy.deepcopy(ssl_gpg_sig[other_headers]),
# 'signature': copy.depcopy(ssl_gpg_sig['signature'])
# }