FIX: broken onebox images due to escape_uri bugs #17840
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
ZogStriP
approved these changes
Aug 9, 2022
lib/url_helper.rb
Outdated
| url = uri.to_s | ||
|
|
||
| # edge case where we expect mailto:test%40test.com to normalize to mailto:test@test.com | ||
| if url.match?(/\A#{URI::regexp}\z/) && !url.match(/\Amailto/) |
There was a problem hiding this comment.
/\Amailto/ would catch "mailto.example.com". We should check for the colon as well
Suggested change
| if url.match?(/\A#{URI::regexp}\z/) && !url.match(/\Amailto/) | |
| if url.match?(/\A#{URI::regexp}\z/) && !url.match(/\Amailto:/) |
will push a patch 👀
There was a problem hiding this comment.
Unfortunately, matching URI::regexp does not guarantee that the URL can be parsed.
pry(main)> URI::regexp.match?("https://éxample.com")
=> true
pry(main)> URI.parse("https://éxample.com")
URI::InvalidURIError: URI must be ascii only "https://\u00E9xample.com"
I think we should try URI.parse, and then fallback to addressable if an exception is raised. Will push a patch 👀
normalized_encode in addressable has a number of issues, including sporkmonger/addressable#472 To temporaily work around those issues for the majority of cases, we try parsing with `::URI`. If that fails (e.g. due to non-ascii characters) then we will fall back to addressable. Hopefully we can simplify this back to `Addressable::URI.normalized_encode` in the future. This commit also adds support for unicode domain names and emoji domain names with escape_uri. This removes an unneeded hack checking for pre-signed urls, which are now handled by the general case due to starting off valid and only being minimally normalized. Previous test case continues to pass. UrlHelper.s3_presigned_url? which was somewhat wide was removed.
This is a much better description of its function. It performs idempotent normalization of a URL. If consumers truly need to `encode` a URL (including double-encoding of existing encoded entities), they can use the existing `.encode` method.
davidtaylorhq
force-pushed
the
normalize-implementation
branch
from
b29e7ab
to
6062c9a
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
normalized_encode in addressable is buggy due to:
sporkmonger/addressable#472
New implementation avoids any escaping (and only performs basic normalization)
if URL is already valid.
Vast majority to the calls to escape_uri start with valid urls.
This also leaves an edge case around "part" escaped urls, where some chars
are escaped and others are not. In those cases addressable may corrupt stuff.
Also added support for unicode domain names and emoji domain names
with escape uri
This removes an uneeded hack checking for pre-signed urls, which are now
handled by the general case due to starting off valid and only being minimally
normalized. Previous test case continues to pass.
UrlHelper.s3_presigned_url? which was somewhat wide was removed.