Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Validate convert entity code points #579

Merged
merged 2 commits into from Jan 26, 2021
Merged

Validate convert entity code points #579

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
6879f6a
html5lib_shim: validate unicode points for convert_entity
Jan 25, 2021
612b808
Update for v3.2.3 release
Jan 25, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
17 changes: 16 additions & 1 deletion CHANGES
View file
Open in desktop
@@ -1,8 +1,23 @@
Bleach changes
==============

Version 3.2.3 (January 26th, 2021)
----------------------------------

**Security fixes**

None

**Features**

None

**Bug fixes**

* fix clean and linkify raising ValueErrors for certain inputs. Thank you @Google-Autofuzz.

Version 3.2.2 (January 20th, 2021)
------------------------------------
----------------------------------

**Security fixes**

Expand Down
4 changes: 2 additions & 2 deletions bleach/__init__.py
View file
Open in desktop
Expand Up @@ -18,9 +18,9 @@


# yyyymmdd
__releasedate__ = "20210120"
__releasedate__ = "20210126"
# x.y.z or x.y.z.dev0 -- semver
__version__ = "3.2.2"
__version__ = "3.2.3"
VERSION = packaging.version.Version(__version__)


Expand Down
17 changes: 15 additions & 2 deletions bleach/html5lib_shim.py
View file
Open in desktop
Expand Up @@ -459,9 +459,22 @@ def convert_entity(value):
if value[0] == "#":
if len(value) < 2:
return None

if value[1] in ("x", "X"):
return six.unichr(int(value[2:], 16))
return six.unichr(int(value[1:], 10))
# hex-encoded code point
int_as_string, base = value[2:], 16
else:
# decimal code point
int_as_string, base = value[1:], 10

if int_as_string == "":
return None

code_point = int(int_as_string, base)
if 0 < code_point < 0x110000:
return six.unichr(code_point)
else:
return None

return ENTITIES.get(value, None)

Expand Down
10 changes: 10 additions & 0 deletions tests/test_html5lib_shim.py
View file
Open in desktop
Expand Up @@ -19,6 +19,16 @@
("&xx;", "&xx;"),
# Handles multiple entities in the same string
("this &amp; that &amp; that", "this & that & that"),
# Handles empty decimal and hex encoded code points
("&#x;", "&#x;"),
("&#;", "&#;"),
# Handles too high unicode points
("&#x110000;", "&#x110000;"),
("&#x110111;", "&#x110111;"),
("&#9277809;", "&#9277809;"),
# Handles negative unicode points
("&#-1;", "&#-1;"),
("&#x-1;", "&#x-1;"),
],
)
def test_convert_entities(data, expected):
Expand Down
4 changes: 2 additions & 2 deletions tests_website/index.html
View file
Open in desktop
Expand Up @@ -2,7 +2,7 @@
<html>
<head>
<meta charset="UTF-8">
<title>Python Bleach 3.2.2</title>
<title>Python Bleach 3.2.3</title>
<style>
textarea, iframe {
width: 95%;
Expand All @@ -20,7 +20,7 @@
</style>
</head>
<body>
<h2>Python Bleach 3.2.2</h2>
<h2>Python Bleach 3.2.3</h2>
<p>
<a href="http://badge.fury.io/py/bleach"><img style="max-width:100%;" alt="pypi version" src="https://badge.fury.io/py/bleach.svg"></a>
<a href="https://github.com/mozilla/bleach/actions?query=workflow%3ATest"><img style="max-width:100%;" alt="Build Status" src="https://github.com/mozilla/bleach/workflows/Test/badge.svg"></a>
Expand Down