Skip to content

Encoding issue throwing exception on 1.15.3 #1873

Closed
@dorfri

Description

@dorfri

Posted also on StackOverflow: https://stackoverflow.com/questions/74917912/url-encoding-in-jsoup-not-working-properly

When I changed version from 1.11.3 to 1.15.3 I started getting MalformedUrlException when fetching URLs with characters that need encoding, like: https://im-creation-assets.s3-us-west-2.amazonaws.com/CelebrityCars[DE]/20221208JuliaRobertsCarJvo/juliayoung-1___native_1200-627.jpg)
Because of the '[' and ']' in the URL...

The exception comes from org.jsoup.helper.CookieUtil#asUri - and was added somewhere between those versions.

I see that your code tries to encode in org.jsoup.helper.HttpConnection#encodeUrl - but this encoding does not work on this URL (and many more).
I can do the encoding myself before calling org.jsoup.Jsoup#connect - the problem is that if there is a redirect to such a URL, I get back to this error.

We do encoding, by the way, using springframework, and it works pretty well, something like:

import org.springframework.web.util.UriComponentsBuilder;
final String encodedUrl = UriComponentsBuilder
.fromUriString(url)
.build()
.encode()
.toUri()
.toString();

(I know I can avoid redirects - do them myself - and encode every URL... but maybe it is possible to fix this encoding issue for everyone...).

Thanks!

Activity

self-assigned this
on Jan 5, 2023
added
bugA confirmed bug, that we should fix
on Jan 5, 2023
added this to the 1.15.4 milestone on Jan 5, 2023
added
fixedAn {bug|improvement} that has been {fixed|implemented}
on Jan 5, 2023
jhy

jhy commented on Jan 5, 2023

@jhy
Owner

Thanks for the report! Fixed with 45ed002, will be in the next release.

added a commit that references this issue on Jan 5, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

bugA confirmed bug, that we should fixfixedAn {bug|improvement} that has been {fixed|implemented}

Projects

No projects

Relationships

None yet

    Development

    No branches or pull requests

      Participants

      @jhy@dorfri

      Issue actions

        Encoding issue throwing exception on 1.15.3 · Issue #1873 · jhy/jsoup