python-bindings: Add pickle support for G1Element
#265
Conversation
| seed = bytes([ | ||
| 0, 50, 6, 244, 24, 199, 1, 25, 52, 88, 192, 19, 18, 12, 89, 6, | ||
| 220, 18, 102, 58, 209, 82, 12, 62, 89, 110, 182, 9, 44, 20, 254, 22 | ||
| ]) |
There was a problem hiding this comment.
With this change, This binary format is part of our state that would need to be migrated if it ever changes. Is pickle a stable format? If it can change between python versions it seems risky to depend on.
There should probably be a comment here saying that these bytes may not change. If they do we'll have a migration task as well.
There was a problem hiding this comment.
This is only used as parameter for AugSchemeMPL.key_gen to generate a PrivateKey. I don't understand your concerns about changing bytes here. Can you elaborate?
There was a problem hiding this comment.
Oh, I didn't read this very carefully :/
I was reading this as the pickled bytes and the test to ensure it was de-pickled correctly. Since the binary format for this needs to be stable, I think a test like that would be good. It would alert us of any changes that alters the file format.
There was a problem hiding this comment.
There was a problem hiding this comment.
Hmm dropped it again because it appears to create different results on CI for whatever reason.. i spent too much time with this stuff already and i don't think its super important to have this test but if you have something which works locally and on CI feel free to provide a commit.
There was a problem hiding this comment.
You might consider a few more tests of round-trips, especially of the infinity value (since it serializes in a unique way).
xdustinface
force-pushed
the
pr-pickle-support
branch
from
20d3768
to
4740720
Compare
This is needed for plots caching in `chia-blockchain`.
xdustinface
force-pushed
the
pr-pickle-support
branch
from
4740720
to
50b4ff6
Compare
There was a problem hiding this comment.
I'm not an expert on pybind, so I can only say this looks reasonable.
A couple comments about using picking in general: it gives me a bad feeling to have chia-blockchain depend on yet another serialization mechanism, especially one so python-specific, and especially that it's used in persistence. At least it's just cache, so if we later reimplement this part in another language that doesn't easily support pickled data structures, we could get away with just ditching the python-based cache and rebuild it (but we'd likely use the Streamable binary or JSON form at that point anyway, so why not just do that now?).
Also, I'm a bit puzzled by the fact that G1Element doesn't already support pickling. Aren't we pickling a large number of our data structures when we pass them across the multiprocessing pool boundary? How is that working? (In this case, I'm slightly less worried about pickling, since it's a private implementation detail.)
|
My concerns above notwithstanding, I still approve this PR since it seems unlikely that adding pickle-ability has any downsides even if we don't end up using it ourselves. |
…etwork#265)" This reverts commit 616230c.
This is needed for plots caching in `chia-blockchain`.
…etwork#265)" (Chia-Network#278) This reverts commit 616230c.
This is needed for plots caching in
chia-blockchain.