Switch to base64 crate
#1294
Switch to base64 crate
#1294
Conversation
This removes our implementation of base64 encoding/decoding from `aws-smithy-types`, switching to the `base64` Rust crate. Our implementation was written at a time when we wanted to keep dependencies at an absolute minimum. However, a performant base64 implementation is now needed for the server. Our implementation was also accepting invalid base64 (see #1277). Fixes #1277.
| [[smithy-rs]] | ||
| message = "Switch to `base64` crate" | ||
| references = ["smithy-rs#1294"] | ||
| meta = { "breaking" = true, "tada" = false, "bug" = false } |
There was a problem hiding this comment.
Where do I add this message?
The base64 implementation from
aws-smithy-typeshas been removed. Please switch to another implementation (e.g. thebase64crate).
I wanted to add it to aws/SDK_CHANGELOG.md, but that file is autogenerated from update-changelog from tools/sdk-lints. How are the migration guides from that file added if the file is autogenerated? Do they get added manually just before preparing a release?
There was a problem hiding this comment.
There's an example above—[[aws-sdk-rust]] instead of smithy-rs.
Anything you want in the message goes in the message field—TOML allows multiline strings. However: I think we should keep the same function signatures and just delegate to the base64 crate behind the scenes. Having a "blessed" base64 implementation inside of aws-smithy-types is probably generally a good thing to have.
There was a problem hiding this comment.
There's an example above—[[aws-sdk-rust]] instead of smithy-rs.
But how do I add a longer migration guide e.g. https://github.com/awslabs/smithy-rs/blob/775d227c5fbc46c2e3c2878b270b8e8226ab7ef1/aws/SDK_CHANGELOG.md?plain=1#L5-L33 ? Do I plop all of that Markdown into message?
However: I think we should keep the same function signatures and just delegate to the base64 crate behind the scenes. Having a "blessed" base64 implementation inside of aws-smithy-types is probably generally a good thing to have.
I thought a bit about this before making the change and concluded that there's no benefit. base64 is standard and I can't foresee us or Smithy wanting to customize anything that is not already customizable via the base64 crate. I also don't think that SDK users should have public access to a base64-encoding routine from a runtime crate; they should have no need for it after all, since they should just be using aws_smithy_types::Blob. I think having a "blessed" base64 implementation is unnecessarily increasing public API surface.
There was a problem hiding this comment.
Maybe blob should add a from_base64 method...
There was a problem hiding this comment.
yeah, just use triple quote strings for long messages
There was a problem hiding this comment.
Ok, I've added it here f1c00cb
|
A new generated diff is ready to view. |
Rust Wrk benchmark report:Duration: 90 sec, Connections: 32, Threads: 2
|
| [[smithy-rs]] | ||
| message = "Switch to `base64` crate" | ||
| references = ["smithy-rs#1294"] | ||
| meta = { "breaking" = true, "tada" = false, "bug" = false } |
There was a problem hiding this comment.
There's an example above—[[aws-sdk-rust]] instead of smithy-rs.
Anything you want in the message goes in the message field—TOML allows multiline strings. However: I think we should keep the same function signatures and just delegate to the base64 crate behind the scenes. Having a "blessed" base64 implementation inside of aws-smithy-types is probably generally a good thing to have.
|
Let's hold off merging this until we know Smithy's firm stance on extant base64-encoded data. |
|
@david-perez smithy-lang/smithy#1168 was merged—what's the plan here? |
|
@rcoh It was reverted, and for good reason; see smithy-lang/smithy#1168 (comment). To move forward; we either wait for/implement marshallpierce/rust-base64#181, or we switch to another crate that rejects unpadded inputs, like base64ct. |
|
We also just fix our impl for now |
|
Hi. The base64-simd crate may be what you want. |
|
Superseded by #1938. |
This removes our implementation of base64 encoding/decoding from
aws-smithy-types, switching to thebase64Rust crate.Our implementation was written at a time when we wanted to keep
dependencies at an absolute minimum. However, a performant base64
implementation is now needed for the server.
Our implementation was also accepting invalid base64 (see #1277).
Fixes #1277.
Checklist
CHANGELOG.next.tomlif I made changes to the smithy-rs codegen or runtime cratesCHANGELOG.next.tomlif I made changes to the AWS SDK, generated SDK code, or SDK runtime cratesBy submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.