Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Sanitize hardening #1504

Merged
merged 9 commits into from Jul 2, 2019
Merged

Sanitize hardening #1504

Show file tree
Hide file tree
Changes from 8 commits
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
6b4d792
add deprecation warning for sanitize option
koczkatamas Jun 26, 2019
08389db
harden sanitization
koczkatamas Jun 26, 2019
506704a
add test cases for sanitization hardening
koczkatamas Jun 26, 2019
0c38172
add sanitize deprecation warnings to docs
koczkatamas Jun 26, 2019
73ad658
make lint happy
koczkatamas Jun 26, 2019
44da69e
recommend more variety of sanitize libraries
koczkatamas Jun 28, 2019
0e8d8f4
use the more generic `sanitizer` option instead of `sanitizerRemoveHt…
koczkatamas Jun 28, 2019
63fd313
fix insane's URL
koczkatamas Jun 28, 2019
b78e498
update deprecation warning
UziTech Jul 2, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
2 changes: 1 addition & 1 deletion README.md
View file
Open in desktop
Expand Up @@ -39,7 +39,7 @@ Also read about:

## Usage

### Warning: 🚨 Marked does not [sanitize](https://marked.js.org/#/USING_ADVANCED.md#options) the output HTML by default 🚨
### Warning: 🚨 Marked does not [sanitize](https://marked.js.org/#/USING_ADVANCED.md#options) the output HTML. Please use a sanitize library, like [DOMPurify](https://github.com/cure53/DOMPurify) (recommended), [sanitize-html](https://github.com/apostrophecms/sanitize-html) or [insane](https://github.com/bevacqua/insane) on the output HTML! 🚨
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we have this list in one place and just link to it. I feel like it will be hard to keep these lists in sync if we ever have to update them.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Any suggestion where would you like to put that list exactly? It's your project afterall and I feel like this is the point where this change has nothing to do with the security fix.

Also there are already two README.mds (/README.md and /docs/README.md) with similar content, so you have to keep those in sync too...

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@koczkatamas: Thank you so much for this!

@styfle: Where would the best place be to store this list? Thinking it would be a nice list for users moving forward as we are planning to get out of the sanitizer business and working looking for things like this anyway. (I have a note to catch up on what I've missed with all the weirdness of life soon.)


**CLI**

Expand Down
2 changes: 1 addition & 1 deletion docs/README.md
View file
Open in desktop
Expand Up @@ -25,7 +25,7 @@ These documentation pages are also rendered using marked 💯

<h2 id="usage">Usage</h2>

### Warning: 🚨 Marked does not [sanitize](https://marked.js.org/#/USING_ADVANCED.md#options) the output HTML by default 🚨
### Warning: 🚨 Marked does not [sanitize](https://marked.js.org/#/USING_ADVANCED.md#options) the output HTML. Please use a sanitize library, like [DOMPurify](https://github.com/cure53/DOMPurify) (recommended), [sanitize-html](https://github.com/apostrophecms/sanitize-html) or [insane](https://github.com/bevacqua/insane) on the output HTML! 🚨

**CLI**

Expand Down
2 changes: 1 addition & 1 deletion docs/USING_ADVANCED.md
View file
Open in desktop
Expand Up @@ -51,7 +51,7 @@ console.log(marked(markdownString));
|mangle |`boolean` |`true` |v0.3.4 |If true, autolinked email address is escaped with HTML character references.|
|pedantic |`boolean` |`false` |v0.2.1 |If true, conform to the original `markdown.pl` as much as possible. Don't fix original markdown bugs or behavior. Turns off and overrides `gfm`.|
|renderer |`object` |`new Renderer()`|v0.3.0|An object containing functions to render tokens to HTML. See [extensibility](USING_PRO.md) for more details.|
|sanitize |`boolean` |`false` |v0.2.1 |If true, sanitize the HTML passed into `markdownString` with the `sanitizer` function.|
|sanitize |`boolean` |`false` |v0.2.1 |If true, sanitize the HTML passed into `markdownString` with the `sanitizer` function.<br>**Warning**: This feature is deprecated and it should NOT be used as it cannot be considered secure.<br>Instead use a sanitize library, like [DOMPurify](https://github.com/cure53/DOMPurify) (recommended), [sanitize-html](https://github.com/apostrophecms/sanitize-html) or [insane](https://github.com/bevacqua/insane) on the output HTML! |
|sanitizer |`function`|`null` |v0.3.4 |A function to sanitize the HTML passed into `markdownString`.|
|silent |`boolean` |`false` |v0.2.7 |If true, the parser does not throw any exception.|
|smartLists |`boolean` |`false` |v0.2.8 |If true, use smarter list behavior than those found in `markdown.pl`.|
Expand Down
12 changes: 10 additions & 2 deletions lib/marked.js
View file
Open in desktop
Expand Up @@ -434,7 +434,7 @@ Lexer.prototype.token = function(src, top) {
: 'html',
pre: !this.options.sanitizer
&& (cap[1] === 'pre' || cap[1] === 'script' || cap[1] === 'style'),
text: cap[0]
text: this.options.sanitize ? (this.options.sanitizer ? this.options.sanitizer(cap[0]) : escape(cap[0])) : cap[0]
Copy link
Contributor

@davisjam davisjam Jun 29, 2019 •
edited

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will be a semantic change for anyone using this feature, right? Same comment on the similar line below.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep, for example if their code allowed XSS then now it won't. 😉

There are no test cases on how the sanitizer should work, so basically any previous change could change this feature also.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What I mean is, if anyone is currently invoking marked with the sanitize option, the output may change if we land this PR.

  1. Yes, they will be (less) XSS-able
  2. But since we didn't really document what "sanitize" did, they might have enjoyed its previous behavior (e.g. assuming that by sanitize we meant removing unclosed HTML tags or similar).

As a result I'm not sure how to semver this.

});
continue;
}
Expand Down Expand Up @@ -847,7 +847,7 @@ InlineLexer.prototype.output = function(src) {
if (cap = this.rules.text.exec(src)) {
src = src.substring(cap[0].length);
if (this.inRawBlock) {
out += this.renderer.text(cap[0]);
out += this.renderer.text(this.options.sanitize ? (this.options.sanitizer ? this.options.sanitizer(cap[0]) : escape(cap[0])) : cap[0]);
} else {
out += this.renderer.text(escape(this.smartypants(cap[0])));
}
Expand Down Expand Up @@ -1536,6 +1536,12 @@ function findClosingBracket(str, b) {
return -1;
}

function checkSanitizeDeprecation(opt) {
if (opt && opt.sanitize && !opt.silent) {
console.warn('marked(): sanitize and sanitizer parameters are deprecated since version 0.6.3, should not be used and will be removed in the next major version. Read more here: https://marked.js.org/#/USING_ADVANCED.md#options');
}
}

/**
* Marked
*/
Expand All @@ -1557,6 +1563,7 @@ function marked(src, opt, callback) {
}

opt = merge({}, marked.defaults, opt || {});
checkSanitizeDeprecation(opt);

var highlight = opt.highlight,
tokens,
Expand Down Expand Up @@ -1621,6 +1628,7 @@ function marked(src, opt, callback) {
}
try {
if (opt) opt = merge({}, marked.defaults, opt);
checkSanitizeDeprecation(opt);
return Parser.parse(Lexer.lex(src, opt), opt);
} catch (e) {
e.message += '\nPlease report this to https://github.com/markedjs/marked.';
Expand Down
5 changes: 5 additions & 0 deletions test/specs/run-spec.js
View file
Open in desktop
Expand Up @@ -16,6 +16,10 @@ function runSpecs(title, dir, showCompletionTable, options) {
spec.options = Object.assign({}, options, (spec.options || {}));
const example = (spec.example ? ' example ' + spec.example : '');
const passFail = (spec.shouldFail ? 'fail' : 'pass');
if (spec.options.sanitizer) {
// eslint-disable-next-line no-eval
spec.options.sanitizer = eval(spec.options.sanitizer);
}
koczkatamas marked this conversation as resolved.
Show resolved Hide resolved
(spec.only ? fit : it)('should ' + passFail + example, () => {
const before = process.hrtime();
if (spec.shouldFail) {
Expand All @@ -40,3 +44,4 @@ runSpecs('CommonMark', './commonmark', true, { headerIds: false });
runSpecs('Original', './original', false, { gfm: false });
runSpecs('New', './new');
runSpecs('ReDOS', './redos');
runSpecs('Security', './security', false, { silent: true }); // silent - do not show deprecation warning
6 changes: 6 additions & 0 deletions test/specs/security/sanitizer_bypass.html
View file
Open in desktop
@@ -0,0 +1,6 @@
<p>AAA&lt;script&gt; &lt;img &lt;script&gt; src=x onerror=alert(1) /&gt;BBB</p>

<p>AAA&lt;sometag&gt; &lt;img &lt;sometag&gt; src=x onerror=alert(1)BBB</p>

<p>&lt;a&gt;a2&lt;a2t&gt;a2&lt;/a&gt; b &lt;c&gt;c&lt;/c&gt; d</p>
<h1 id="text"><img src="URL" alt="text"></h1>
9 changes: 9 additions & 0 deletions test/specs/security/sanitizer_bypass.md
View file
Open in desktop
@@ -0,0 +1,9 @@
---
sanitize: true
---
AAA<script> <img <script> src=x onerror=alert(1) />BBB

AAA<sometag> <img <sometag> src=x onerror=alert(1)BBB

<a>a2<a2t>a2</a> b <c>c</c> d
# ![text](URL)
2 changes: 2 additions & 0 deletions test/specs/security/sanitizer_bypass_remove_generic.html
View file
Open in desktop
@@ -0,0 +1,2 @@
<p>a2a2 b c d</p>
<h1 id="text"><img src="URL" alt="text"></h1>
6 changes: 6 additions & 0 deletions test/specs/security/sanitizer_bypass_remove_generic.md
View file
Open in desktop
@@ -0,0 +1,6 @@
---
sanitize: true
sanitizer: () => ''
---
<a>a2<a2t>a2</a> b <c>c</c> d
# ![text](URL)
1 change: 1 addition & 0 deletions test/specs/security/sanitizer_bypass_remove_script.html
View file
Open in desktop
@@ -0,0 +1 @@
<p>AAA</p>
5 changes: 5 additions & 0 deletions test/specs/security/sanitizer_bypass_remove_script.md
View file
Open in desktop
@@ -0,0 +1,5 @@
---
sanitize: true
sanitizer: () => ''
---
AAA<script> <img <script> src=x onerror=alert(1) />BBB
1 change: 1 addition & 0 deletions test/specs/security/sanitizer_bypass_remove_tag.html
View file
Open in desktop
@@ -0,0 +1 @@
<p>AAA &lt;img src=x onerror=alert(1)BBB</p>
5 changes: 5 additions & 0 deletions test/specs/security/sanitizer_bypass_remove_tag.md
View file
Open in desktop
@@ -0,0 +1,5 @@
---
sanitize: true
sanitizer: () => ''
---
AAA<sometag> <img <sometag> src=x onerror=alert(1)BBB