Add support for skipping wildcard URLs #38
Conversation
URLs can contain wildcards as in http://*.example.org. These are commonly used in cases where listing all possible URLs would not be feasible, e.g. when defining Content-Security-Policy rules [1]. It can be surprising to users of linkify that these URLs get extracted. As such, this change adds support for disabling the extraction of wildcard URLs. [1]: https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Content-Security-Policy
mre
changed the title
There was a problem hiding this comment.
Hmm, I'm not sure the current solution is the way to go about this, see inline comment.
But even if that inline comment was fixed, I'm not even sure we'd need an option, we should never allow * in the domain (I haven't checked the RFC but pretty sure it's not allowed). E.g. GitHub also doesn't linkify those kinds of URLs, see here: https://*.example.org (but it allows https://a*.example.org)
I feel like this is another one of those cases where it would be good to have stricter parsing of domains, as it would also address some other issues, e.g. #28 (comment).
I think I don't want to accept this in the current state, I'd rather clean up domain parsing which should hopefully make this change unnecessary.
| @@ -162,6 +164,11 @@ impl UrlScanner { | |||
| // These may be part of an URL but not at the end. It's not that the spec | |||
| // doesn't allow them, but they are frequently used in plain text as delimiters | |||
| // where they're not meant to be part of the URL. | |||
| if !extract_wildcard_urls { | |||
There was a problem hiding this comment.
I don't think this is the right place to add this. If using .wildcards(false) and the text https://example.org/foo.bar, it would only extract https://example.org/foo.
Sure, no worries. Gonna close this PR for now. Would be glad if you could look into the domain parsing if you find some time. 👍 |
Came up in a couple of places: #41, #29, #38, #28. Hopefully we can fix all of these with these changes. Not done yet, still want to have domain checking for URLs with certain schemes (https) but allow everything for others. If we do that, we may be able to unify the email and plain domain parsing with the scheme one too.
|
With the changes in 0.9.0, this is no longer a problem: https://github.com/robinst/linkify/blob/main/CHANGELOG.md#090---2022-07-11 |
URLs can contain wildcards as in http://*.example.org.
These are commonly used in cases where listing all possible URLs would
not be feasible, e.g. when defining Content-Security-Policy rules 1.
It can be surprising to users of linkify that these URLs get extracted.
As such, this change adds support for disabling the extraction of wildcard
URLs.
Fixes #37.